From xen-users-bounces@lists.xenproject.org Tue Dec 03 18:34:21 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 03 Dec 2024 18:34:21 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848111.1263102 (Exim 4.92) (envelope-from ) id 1tIXiI-000266-Ul; Tue, 03 Dec 2024 18:33:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848111.1263102; Tue, 03 Dec 2024 18:33:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIXiI-00025z-Ro; Tue, 03 Dec 2024 18:33:34 +0000 Received: by outflank-mailman (input) for mailman id 848111; Tue, 03 Dec 2024 18:33:33 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIXiH-00025t-LQ for xen-users@lists.xenproject.org; Tue, 03 Dec 2024 18:33:33 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 171dc0b2-b1a5-11ef-a0d3-8be0dac302b0; Tue, 03 Dec 2024 19:33:31 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 171dc0b2-b1a5-11ef-a0d3-8be0dac302b0 Date: Tue, 3 Dec 2024 18:30:43 +0000 From: Mike To: xen-users@lists.xenproject.org Subject: CPU oversubscription =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Has anyone noticed their Xen system spontaneously rebooting itself? And did that correlate with your overcommitting the vCPUs, by handing out more than the available number, via the `dom0_max_vcpus` boot param and the `maxvcpus` domU config param? I've been experiencing spontaneous reboots of my Xen system lately, and it seems to correlate with my increasing use of the CPU. Particularly, it seems to happen right after I hit the return key to execute a shell command. The reboot is sudden: I hear a slight click from the hardware and then I see the BIOS loading. So I'm casting about for an answer as to what might cause such a low-level failure. Xen or a kernel bug seem like the most likely choices. The hardware itself has been otherwise well-behaved in the six months or so that I've had it running, except for some very early reboots that I attribute vaguely to "burn-in". The above matter is drawing my suspicion currently, so to start with I added `dom0_vcpus_pin` to the boot line and `cpus = "all,^0-3"` to the domU configs for now. From xen-users-bounces@lists.xenproject.org Tue Dec 03 22:27:13 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 03 Dec 2024 22:27:13 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848222.1263177 (Exim 4.92) (envelope-from ) id 1tIbLk-0006U7-I2; Tue, 03 Dec 2024 22:26:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848222.1263177; Tue, 03 Dec 2024 22:26:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIbLk-0006U0-FO; Tue, 03 Dec 2024 22:26:32 +0000 Received: by outflank-mailman (input) for mailman id 848222; Tue, 03 Dec 2024 22:26:30 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIbLi-0006Tu-JQ for xen-users@lists.xenproject.org; Tue, 03 Dec 2024 22:26:30 +0000 Received: from CWXP265CU009.outbound.protection.outlook.com (mail-ukwestazlp170110003.outbound.protection.outlook.com [2a01:111:f403:c206::3]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a3b2853d-b1c5-11ef-a0d4-8be0dac302b0; Tue, 03 Dec 2024 23:26:28 +0100 (CET) Received: from LO6P265MB6141.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2b9::6) by CW1P265MB7692.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:21f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.20; Tue, 3 Dec 2024 22:26:26 +0000 Received: from LO6P265MB6141.GBRP265.PROD.OUTLOOK.COM ([fe80::8484:50c3:e372:2799]) by LO6P265MB6141.GBRP265.PROD.OUTLOOK.COM ([fe80::8484:50c3:e372:2799%6]) with mapi id 15.20.8207.017; Tue, 3 Dec 2024 22:26:25 +0000 X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: a3b2853d-b1c5-11ef-a0d4-8be0dac302b0 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=J/m6N2Bx1MuowMK65jYFhucTo3Z5oaMXJttIvAjNRolXJeIyS3qs6X9/nmZJ+ZC8OT6BN/uqz21KPPgwlF+PzHAQN9JjTi3Hp12SbryF+7Ean1NTJiyKNyngT9MlJUDZl3gmCN/GTqTIwGXXo1C/Phw0VRq+S6M+iuXCDHmHp3z43uskexz4DycOPaZfesXz/eQdH5chmp7fuIgnpFFMmUqNwZqnwRB/DNDPYNbIWCv8o+GmojgyRaaOL0VepwRAfFy9vHN7f18S6OxTS1OUjp9LDeYkJuWlgWgatnERmhyeOXSRrR76AAmITbJZYd9GSA9j2O+z5w/IReUDzWAzQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WpA55lsaZ1mj0uRDRpcGUF63E7093ArlZpmL40IZZ7o=; b=rh/KoWhojLPcjlGXMNfo7wswGDmzrrMRoLRYpTvkM0BfRRVXqWU92MEi6W5WI/UhggRMyUGgYF6ZCVZFvs3o2PgO71z2ZTO/wJwiVFSXHi2783dZ007AQzluLtsY0Qr1qMTwEvXigBVwRDS5mNDfJoFk6+4h79Glpv7DYQQZ/UadU/X2V/ZPlw63vIQj6oqbFH3sb2NVhZcdAXjH2LEFAJnOV4+4PbfiIqlKqloHZ5etvgGVqrqactwfa6mKt1u8dJHFhmK8mWheKlseZGJTWzMJfZkNIpQaDeimM7amzCqaEOdEmm2FdbVgFsdVo3s+Ump8Rbj05Kh0wa3H8HBGFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=durham.ac.uk; dmarc=pass action=none header.from=durham.ac.uk; dkim=pass header.d=durham.ac.uk; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=durham.ac.uk; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WpA55lsaZ1mj0uRDRpcGUF63E7093ArlZpmL40IZZ7o=; b=JCi0RYRemjG7S6KpfB8970o1S6c9Pes2+5hTZI7i6WKul0jWVpMb/3hq0LBFDwSkn3RGGbF0jRzCwiPJaPKHjGyXeiN+xwRpWo2s+capy+5lIa8JtWsgzLfZV3rO6dMDHl8U5PuiVN70j9nmIwcp+EXu47tRK31TAha2vdn3EOg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=durham.ac.uk; Date: Tue, 3 Dec 2024 22:26:25 +0000 (GMT) From: Michael Young To: xen-users@lists.xenproject.org Subject: Should firmware='/usr/share/edk2/xen/OVMF.fd' work? Message-ID: <71da5cfb-5e75-51c0-bea5-be13a596c28b@durham.ac.uk> Content-Type: text/plain; format=flowed; charset=US-ASCII X-ClientProxiedBy: LO2P265CA0236.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:b::32) To LO6P265MB6141.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2b9::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P265MB6141:EE_|CW1P265MB7692:EE_ X-MS-Office365-Filtering-Correlation-Id: 05a0ebd6-1191-49ce-39f4-08dd13e98641 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024|10070799003; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Rf2z4g2lV7KCbeA+zxiIFtzFsf75vnzZ4MEswCoCbO04Q7BEQlW58Gk+/s+/?= =?us-ascii?Q?wTqkrrtzxBquWYGm/BKDZP7t8k0bLyCIxTO4lqwdJur23yJde6nMMfxL8aDA?= =?us-ascii?Q?3KpZhLPR4EYVSPhZDX7VUp6b4oL8Su8Z/VrPlxh0f8JrasH1wOfrmHmLb2C1?= =?us-ascii?Q?djb/wU3t71rNlQSaAFfVV0cdW05w+FXx+77bxyjXoxU9GYZ23B4NJG/rELQ2?= =?us-ascii?Q?4jSocYwxXHfL5zx09gZ3kjGhIXwHK+QnZA99XUBumfdMxLZUEykFBZO2WFvV?= =?us-ascii?Q?TOEimVI7PqROVGYsa/PwagTxTFyeZJNtw3305JqpCKNbuoCyuZeRcog8xL7F?= =?us-ascii?Q?y2DJ8YvYz0+7gTbnPXHZhiz1A+VJ+XXpsh4iBWH/pvxvVPQIBeyTkKFBbfGj?= =?us-ascii?Q?KiBYd46dEY/lTgCBzkO/Ycm8wzwKi5SkS9GMyxnwSjDNIdPplzox50WoJ2Qu?= =?us-ascii?Q?QsdibsJqGDk+h6IUm8L5oQXO/8Q+5RGx9RkTltIs58Rjz+Yhlmdt9gEwYF4P?= =?us-ascii?Q?QaHGtFoC0hq1lowbo/f7/iLb5tSNpmfZ1NvlIsuxkUWf9A2npnQhT3xBJ2vP?= =?us-ascii?Q?IiOrCZHe8lU6upBN+WRvbigEyjN0VOQBHgkFRkc6AFpaCwGBy1NC4Hq6MfoG?= =?us-ascii?Q?prQ1+oa3Kl5FX3bGhx5CHqdcAxkGQAVVwO/1LoWTn6AX5X+XpuQ5xnneTOB5?= =?us-ascii?Q?Y73eUWUtuKDHV0RohtHPQdOIbAh7iXhywWoe7EJS8hMUI8ciN9bH6cmWqUXP?= =?us-ascii?Q?pafI53RZD5YTZO3Ux0nudxpIaRce2tE/yNWMmHH+z2OqvnAWM9GEM1hsUJDt?= =?us-ascii?Q?ZORjpLFcjK28wR3G1syE9ptRR8h1YN75go+9mrrlaYZXR5H1nYFLLTqyqD/z?= =?us-ascii?Q?Z6AEOJs2PdCA/AtFH8LbdOnUe7OcWeS2dEmlybqDpouHWWSgJrvkKo0MxzpJ?= =?us-ascii?Q?KWI9QA/CR85SjOOh7x+/scWGAhGfPNqlooTIJejX/eymLnJYkae9QYSSDstP?= =?us-ascii?Q?dKqCkXl7/qcLvmMWpRpPJkbkOWb8SqO5C4G86XHHoX3iO3HcF0eBn30+9C98?= =?us-ascii?Q?Sd6h9WvZAxXd4rMIWTc0wZ/yl1E9CeCvp5WOWqIeOF8tVd3FTMk0QUMBkE1M?= =?us-ascii?Q?HisCnekfa4rndMsBbMpdO0LSoH4UCZcvVIXvnnNjDzop+mkEpgR0YBMCEbuO?= =?us-ascii?Q?5Z7haPQOZcTBMU7sD5a4Xtpf1bbhMykX/V8w/YSNqxfn/quChHo8SQa+j2aH?= =?us-ascii?Q?VgCyIHax77ODLPoG5Z6KYpw4GdBHamZm0WOOyI+aMbN9S9CcMvDdME6EtaCl?= =?us-ascii?Q?mnQWdGtVveFVsj+K/ibYVd97d9KVhJ4RFjtA7nAR/RXGlt4VkUVvxXLPjNH0?= =?us-ascii?Q?lRlw+oY=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P265MB6141.GBRP265.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(10070799003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?U/NjyojObjr/kIPIqLELWjg+Kp5TRsYDIzDvzvJe4q6Skfe8vhtBI+Njpgo1?= =?us-ascii?Q?J7FAt1M1B2MIFb9+cf4zw+RdNV21ELC3rqRJHx01egewDm9fEGq4/cDfSeDC?= =?us-ascii?Q?ECkIbLsrABsxLP+tnRyjrw8J2WpmVl6cNl9Pjpr3vxRpv6pwaDBXJ2S8P52v?= =?us-ascii?Q?GmbIkzW9GudH1MuOAz9kgC7ctinUbCVuVfvhMfsRUVXF8GFPulB+/hI+qYsB?= =?us-ascii?Q?QpUdq1j1LyDz9rcl5FAd1gXY34Fd5HCktVJcMTxWX4S6LgW1O4yAfldHX+NT?= =?us-ascii?Q?8/ahdMQvQiAFn6lPraNxsoRMazxRdpw+Vb33kO6jcxn0ImdVIEnPtegE8ngz?= =?us-ascii?Q?h0zmnpczRLzVb6xftWzWUtbrueihvydul/Gz5oj2M5CuklAaFDal9GO+K6Ef?= =?us-ascii?Q?MfIQJ/odQk7mbEdpf/nq5fOY1eMTQk8qKvDE0zhKMT7afNmSgYDS4XU0Zwfu?= =?us-ascii?Q?o+KgMwJq4hTE39SjFFnDiqLisrSV3A/PBe/57krsqbQW28DbXUFMLZGRjSt3?= =?us-ascii?Q?pljrEKciCS7uWXFd/jqM8WJ/+hWCyZTPFv5ra+Cr39LUoYcPin+RbItJ8YBr?= =?us-ascii?Q?TNI9+/vJ0Ud30Hr7P3+eWtfBEMZNdCH5YI8m0gYlo9q2wcQ1+w/3twXEfRTD?= =?us-ascii?Q?MaCKOyhXfTYhsnepk9Uxm/r9v5cpObaXm869gx3AM1ZhFceUaAb+YFR05kkF?= =?us-ascii?Q?fXeSiNJTdbiwsblhh3dqalL5La4wE+prrRqzoLWQG2DXnjpJACf+u1RUzzcb?= =?us-ascii?Q?I3UG7F+ZLDcv0NWDBuxahG+qxpM32ocOYi+h2yx069876LG/aBhMgWL5STXq?= =?us-ascii?Q?zMN6eAWVeObmlKxeTI51/411CjG14UBnIONTkA6SHdFpxDRsEiBEVfLbt3Xs?= =?us-ascii?Q?DRgk8orpfa00RgdYDzPsgdL7ca7HIFeoTAHK5rUeIWrkbaR3OOhhuaFx23DG?= =?us-ascii?Q?peClDP4SnnNSLDweoVIghNa+BWR2vMlirkC5I0S/E/6NwLMtZXfVuXOxvmy+?= =?us-ascii?Q?cOjVRdA+fjqyHk9xRFXOEp3aahXSU8nb67HX15LRztlSh7PLYhyAc4iyz2De?= =?us-ascii?Q?u/cmExX8+/3W9xGFdNys90rXeU7M9sWHFcPCpXGu6BQ2w0Z04fDNI9tuBpRq?= =?us-ascii?Q?Zp26Wk2t1hyGjIqcplcl1RGvTOjIodUvkAsDa6Uk+Kqa/Px3QO4KOpMqmPm6?= =?us-ascii?Q?AZZfsWuwqBva5QkhpkNra/vSTu9JspNjlaWglWWxQTOfuli9ej/0/vQN9Fze?= =?us-ascii?Q?zlzt5j2q+nyO3GJSs9S4VPUa/HkSivfK47RHbSBaWqX+8IKtyDLkJ7GdyWhz?= =?us-ascii?Q?5Hj5awCkpPzd3jytNvjQWmVxFeXtnMFwEKAdQedSyNsDvPkEJdmug6CQ5w1q?= =?us-ascii?Q?qObtyPgKNBLj84FkRaf8FrzEamLpbT4JcSRgX3/TxpnsNRoMmwx8WUM/9LTq?= =?us-ascii?Q?VwYlXwFUYspusg3n9PSfljhrkEdN8xWyXGcyxuerjOUrw1MQr3mthGmcmsML?= =?us-ascii?Q?14ZW5JoATjKQy74wwdORKz/xeVlauHQfEN8SiyUNfSX+0tIw4w8a0r4w2DFO?= =?us-ascii?Q?fw8h/WthsmqyBwDIyV2bUrOzqKfWQj03hGAYSKSB7Kd+J3shZvOSTVeyZC+A?= =?us-ascii?Q?cf58KLaa/X39oc//8EQ4mkdcmBvZ29ZJ/lI899z+6wWF?= X-OriginatorOrg: durham.ac.uk X-MS-Exchange-CrossTenant-Network-Message-Id: 05a0ebd6-1191-49ce-39f4-08dd13e98641 X-MS-Exchange-CrossTenant-AuthSource: LO6P265MB6141.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2024 22:26:25.8335 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 7250d88b-4b68-4529-be44-d59a2d8a6f94 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XXYH0PEAeztjXWhrWHaRFcdqTl+1KZL+jDtBxCMANCH3gyMRmhM34luGgftXO7GcnnRR4dTOn8jRkfNQS1j5Og== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CW1P265MB7692 I have been testing an issue Fedora bug report https://bugzilla.redhat.com/show_bug.cgi?id=2312032 and if I boot a guest with the configuration type = "hvm" name = "uefitest" memory = 1024 vcpus = 1 disk = [ 'file:/usr/share/edk2/ovmf/UefiShell.iso,sda,r' ] vnc = 1 bios = 'ovmf' serial='pty' I get the UEFI Interactive Shell v2.2 as expected, whereas if I boot with configuration type = "hvm" name = "uefitest" memory = 1024 vcpus = 1 disk = [ 'file:/usr/share/edk2/ovmf/UefiShell.iso,sda,r' ] vnc = 1 firmware = '/usr/share/edk2/xen/OVMF.fd' serial='pty' I don't get any output. So is the latter configuration expected to work? /usr/share/edk2/ovmf/UefiShell.iso is from the edk2-ovmf-20241117-5.fc41 fedora package and /usr/share/edk2/xen/OVMF.fd is from the edk2-ovmf-xen-20241117-5.fc41 package. xen on fedora is built with the --with-system-ovmf=/usr/share/edk2/xen/OVMF.fd option. Michael Young From xen-users-bounces@lists.xenproject.org Tue Dec 03 22:30:26 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 03 Dec 2024 22:30:26 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848244.1263193 (Exim 4.92) (envelope-from ) id 1tIbPQ-0008Mb-53; Tue, 03 Dec 2024 22:30:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848244.1263193; Tue, 03 Dec 2024 22:30:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIbPQ-0008MS-2U; Tue, 03 Dec 2024 22:30:20 +0000 Received: by outflank-mailman (input) for mailman id 848244; Tue, 03 Dec 2024 22:30:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIbPO-0008MK-Fc for xen-users@lists.xenproject.org; Tue, 03 Dec 2024 22:30:18 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 29bd7ff3-b1c6-11ef-99a3-01e77a169b0f; Tue, 03 Dec 2024 23:30:15 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 29bd7ff3-b1c6-11ef-99a3-01e77a169b0f Date: Tue, 3 Dec 2024 22:27:28 +0000 From: Mike To: xen-users@lists.xenproject.org Subject: Re: CPU oversubscription =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: > I added > `dom0_vcpus_pin` to the boot line and `cpus = "all,^0-3"` to the domU configs That didn't solve it. One of the first commands that I try to execute in one domU, which causes activity in the other domU, triggers a reboot. As if the change increased the likeliness of the event. So I pinned each of the domUs' vCPUs: one with `cpus = "4-5"` and the other with `cpus = "all,^0-5"`. Also reduced `vcpus` and `maxvcpus` in the latter, to avoid oversubscription. Testing... I still notice in the latter domU that it assigns multiple vCPUs to a single physical CPU, even when there are enough physical ones to avoid that. Odd. From xen-users-bounces@lists.xenproject.org Wed Dec 04 08:17:29 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Wed, 04 Dec 2024 08:17:29 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848327.1263244 (Exim 4.92) (envelope-from ) id 1tIkYt-0001Zr-Fe; Wed, 04 Dec 2024 08:16:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848327.1263244; Wed, 04 Dec 2024 08:16:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIkYt-0001Zk-D3; Wed, 04 Dec 2024 08:16:43 +0000 Received: by outflank-mailman (input) for mailman id 848327; Wed, 04 Dec 2024 08:16:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIkYr-0001Ze-Tt for xen-users@lists.xenproject.org; Wed, 04 Dec 2024 08:16:42 +0000 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [2a00:1450:4864:20::533]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 155f69c6-b218-11ef-99a3-01e77a169b0f; Wed, 04 Dec 2024 09:16:38 +0100 (CET) Received: by mail-ed1-x533.google.com with SMTP id 4fb4d7f45d1cf-5d10f713ef0so1090344a12.0 for ; Wed, 04 Dec 2024 00:16:38 -0800 (PST) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 155f69c6-b218-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leftsock.com; s=google; t=1733300197; x=1733904997; darn=lists.xenproject.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=tISB4OEkqUshiYJtoqAdVoevdczyRSDGwPCNNlvewa4=; b=AHf909V8LYN8/QNu/b/b447n/ivV4jAl4BM4LsPfYKUkNuQYeSp6cWj9Pmt7zpjaVv GsUZTs33J9zjCJbY7XVcWOhbLOoXbmzpsAEtNc34EVXn6Bab5wiAOPMhlbelSA+zXTcp KdBVK2baMzhw+xJ7K3CjbW7puU+J/n9OpBTS0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733300197; x=1733904997; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=tISB4OEkqUshiYJtoqAdVoevdczyRSDGwPCNNlvewa4=; b=HHLOjk6qDK5dspEt361+ulZijgTxrpraW6fuLY1iNZM6/1omv0CcYZaNgJKd6TnpiL ehGzgtHUWTQV92yNbJ/2jlWinFMsInXiDJLmErmFmJJPeyyzTMODlDDJtvkjjZXMhTy9 ESy+5MprFHSDf/+E2d7mcsPXRtPTL3ZmRz85/+dGAZloM59KgyRxzkhQxvrj+k9Z2JQO NvZY0EYtCDTCrVpaLlOD5cl+zzwjazuR1b7zrAJKxxKYBKSM5EGl4zYfBefWP00VDpAw sT9if1YftP65ZsKh+XJDA0xVKQTiIkPVtf7Blk5GyOp6lmKLeGK/PwuwMDc3m18OIeVk YS0A== X-Gm-Message-State: AOJu0YzHTA7pI/vBGwAuF7hNDfu2M+pr4QJHKFepKZFPuTzHJ5kJof4v ZHDpxHAwfvSP/oi4/76CkCuBrdr+kR0XfobA1FZnDF2ok67pw1tZNm3VV9I4MaeIhCB4KXMycDN eeNbzMO3gB7nMydVo/+ILLGgU6lk9OsXkZGnPL+28V/NtX6Fvzw== X-Gm-Gg: ASbGncu6B3ycYkfYRf5VxMHQXRkVdRG8OJWClAkpk51irShJd6QuDax4x6cpXo7eaaW xXwYMwwNwy4lEKEz30betEgCf5ivCuReA X-Google-Smtp-Source: AGHT+IFUGgLGM80eL88RaP+pDsjDTy2IaF/pF0hQ3SaIZ6lVXBCgLqrkwQPSHHGqx44NoymylKiady2tRoulSovnsGM= X-Received: by 2002:a17:907:7ea6:b0:a9e:82f3:d4ab with SMTP id a640c23a62f3a-aa594535329mr2995529366b.9.1733300196746; Wed, 04 Dec 2024 00:16:36 -0800 (PST) MIME-Version: 1.0 From: Kevin Wang Date: Wed, 4 Dec 2024 00:16:25 -0800 Message-ID: Subject: Ubuntu 24 not starting simple hvm To: xen-users@lists.xenproject.org Content-Type: multipart/mixed; boundary="0000000000008479d706286d66d2" --0000000000008479d706286d66d2 Content-Type: multipart/alternative; boundary="0000000000008479d606286d66d0" --0000000000008479d606286d66d0 Content-Type: text/plain; charset="UTF-8" install fresh Ubuntu server 24 on dell micro-form-factor desktop fgrep Intel /proc/cpuinfo model name : Intel(R) Core(TM) i3-6100T CPU @ 3.20GHz # 2 cores, 4 threads apt-get install xen-hypervisor-amd64 # temporary: setup network on dhcp cat >/etc/netplan/90-xen.yaml <<'__EOF__' network: version: 2 bridges: xenbr0: dhcp4: yes interfaces: [enp2s0] __EOF__ chmod 600 /etc/netplan/90-xen.yaml netplan apply lvcreate --type=linear --name=toolbelt-root -L 40G vg0 # I've used this config across several versions, but this is the first time it hasn't worked. cat >/etc/xen/auto/toolbelt.hvm <<'__EOF__' type = "hvm" name = "toolbelt" memory = "4096" vcpus = 2 vif = [''] disk = ['/dev/vg0/toolbelt-root,raw,hda,w,backendtype=phy','/d/dmg/ubuntu-24.04- live-server-amd64.iso,raw,hdc,r,devtype=cdrom'] #disk = ['phy:/dev/vg0/toolbelt-root,hda,w'] # old format vnc = 1 vnclisten = '127.0.0.1:1' boot="dc" # first cdrom, then bootdisk #boot="cd" # first bootdisk, then cdrom __EOF__ xl -vvvv create /etc/xen/auto/toolbelt.hvm >/tmp/failure.txt 2>&1 # see attached # Problem 1: afaict there's no (decipherable) error. ... and no vm was started xl list Name ID Mem VCPUs State Time(s) Domain-0 0 3888 4 r----- 36.4 After days of tearing my hair out, I finally find a reference to this: cat /var/log/xen/qemu-dm-toolbelt.log xen-qemu-system-i386: -device rtl8139,id=nic0,netdev=net0,mac=00:16:3e:43:bd:b5: failed to find romfile "efi-rtl8139.rom" Problem 2: Can you please add basic debugging instructions? "Xen Common Problems" doesn't have any of this (-vvvv or /var/log/xen/) Problem 3: Where is efi-rtl8139.rom ... Answer 3: apt-get install ipxe-qemu and finally now, after several nights of not being able to find the errors, it boots. Feedback: This is painful. - Kevin --0000000000008479d606286d66d0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
install fresh Ubuntu server 24 on dell micro-form-factor d= esktop

fgrep Intel /proc/cpuinfo
model name =C2=A0 =C2=A0 =C2=A0= : Intel(R) Core(TM) i3-6100T CPU @ 3.20GHz
# 2 cores, 4 threads

a= pt-get install xen-hypervisor-amd64

# temporary: setup network on dh= cp
cat >/etc/netplan/90-xen.yaml <<'__EOF__'
network= :
=C2=A0 =C2=A0version: 2
=C2=A0 =C2=A0bridges:
=C2=A0 =C2=A0 =C2= =A0xenbr0:
=C2=A0 =C2=A0 =C2=A0 =C2=A0dhcp4: yes
=C2=A0 =C2=A0 =C2=A0= =C2=A0interfaces: [enp2s0]
__EOF__
chmod 600 /etc/netplan/90-xen.yam= l
netplan apply

lvcreate --type=3Dlinear --name=3Dtoolbelt-root -= L 40G vg0

# I've used this config across several ver= sions, but this is the first time it hasn't worked.
cat >/etc/xen= /auto/toolbelt.hvm <<'__EOF__'
type =3D "hvm"name =3D "toolbelt"
memory =3D "4096"
vcpus =3D = 2
vif =3D ['']
disk =3D ['/dev/vg0/toolbelt-root,raw,hda,= w,backendtype=3Dphy','/d/dmg/ubuntu-24.04-
live-server-amd64.iso= ,raw,hdc,r,devtype=3Dcdrom']
#disk =3D ['phy:/dev/vg0/toolbelt-r= oot,hda,w'] # old format
vnc =3D 1
vnclisten =3D '127.0.0.1:1'
boot=3D"dc" =C2=A0# f= irst cdrom, then bootdisk
#boot=3D"cd" =C2=A0# first bootdisk,= then cdrom
__EOF__

xl -vvvv create /etc/xen/auto/too= lbelt.hvm >/tmp/failure.txt 2>&1
# see attached
# Problem 1: afaict there's no (decipherable) error.

... and = no vm was started
xl list
Name =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ID =C2=A0 Mem VCPUs =C2=A0 =C2=A0 =C2=A0Stat= e =C2=A0 Time(s)
Domain-0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 0 =C2=A03888 =C2=A0 =C2=A0 4 =C2=A0 =C2=A0 r----- =C2=A0 =C2=A0 =C2= =A036.4

After days of tearing my hair = out, I finally find a reference to this:
cat /var/log/xen/qemu-dm= -toolbelt.log
xen-qemu-system-i386: -device rtl8139,id=3Dnic0,net= dev=3Dnet0,mac=3D00:16:3e:43:bd:b5: failed to find romfile "efi-rtl813= 9.rom"

Problem 2: Can you please add basi= c debugging instructions? "Xen Common Problems" doesn't have = any of this (-vvvv or /var/log/xen/)

Problem 3: Wh= ere is efi-rtl8139.rom ...
Answer 3: apt-get install ipxe-qemu

and finally now, after several nights of not being a= ble to find the errors, it boots. Feedback: This is painful.

=
=C2=A0 =C2=A0- Kevin
--0000000000008479d606286d66d0-- --0000000000008479d706286d66d2 Content-Type: text/plain; charset="US-ASCII"; name="failure.txt" Content-Disposition: attachment; filename="failure.txt" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_m49li69r0 UGFyc2luZyBjb25maWcgZnJvbSAvZXRjL3hlbi9hdXRvL3Rvb2xiZWx0Lmh2bQpsaWJ4bDogZGVi dWc6IGxpYnhsX2NyZWF0ZS5jOjIwODE6ZG9fZG9tYWluX2NyZWF0ZTogYW8gMHg2MWE0Y2E2Njdm ZDA6IGNyZWF0ZTogaG93PShuaWwpIGNhbGxiYWNrPShuaWwpIHBvbGxlcj0weDYxYTRjYTY2ODA3 MApsaWJ4bDogZGV0YWlsOiBsaWJ4bF9jcmVhdGUuYzo2NjI6bGlieGxfX2RvbWFpbl9tYWtlOiBw YXNzdGhyb3VnaDogZGlzYWJsZWQKbGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzo0MTU6bGli eGxfX2RldmljZV9kaXNrX3NldF9iYWNrZW5kOiBEaXNrIHZkZXY9aGRhIHNwZWMuYmFja2VuZD1w aHkgc3BlY2lmaWNhdGlvbj14ZW4KbGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzo0MTU6bGli eGxfX2RldmljZV9kaXNrX3NldF9iYWNrZW5kOiBEaXNrIHZkZXY9aGRjIHNwZWMuYmFja2VuZD1x ZGlzayBzcGVjaWZpY2F0aW9uPXhlbgpsaWJ4bDogZGVidWc6IGxpYnhsX2NyZWF0ZS5jOjEzNDI6 aW5pdGlhdGVfZG9tYWluX2NyZWF0ZTogRG9tYWluIDQ6cnVubmluZyBib290bG9hZGVyCmxpYnhs OiBkZWJ1ZzogbGlieGxfYm9vdGxvYWRlci5jOjQxMTpsaWJ4bF9fYm9vdGxvYWRlcl9ydW46IERv bWFpbiA0Om5vdCBhIFBWL1BWSCBkb21haW4sIHNraXBwaW5nIGJvb3Rsb2FkZXIKbGlieGw6IGRl YnVnOiBsaWJ4bF9ldmVudC5jOjg2MzpsaWJ4bF9fZXZfeHN3YXRjaF9kZXJlZ2lzdGVyOiB3YXRj aCB3PTB4NjFhNGNhNjZkODE4OiBkZXJlZ2lzdGVyIHVucmVnaXN0ZXJlZApkb21haW5idWlsZGVy OiBkZXRhaWw6IHhjX2RvbV9hbGxvY2F0ZTogY21kbGluZT0iIiwgZmVhdHVyZXM9IiIKZG9tYWlu YnVpbGRlcjogZGV0YWlsOiB4Y19kb21fa2VybmVsX2ZpbGU6IGZpbGVuYW1lPSIvdXNyL2xpYi94 ZW4tNC4xNy9ib290L2h2bWxvYWRlciIKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fbWFs bG9jX2ZpbGVtYXAgICAgOiAxNzAga0IKbGlieGw6IGRlYnVnOiBsaWJ4bF9kb20uYzo4MTA6bGli eGxfX2xvYWRfaHZtX2Zpcm13YXJlX21vZHVsZTogTG9hZGluZyBCSU9TOiAvdXNyL3NoYXJlL3Nl YWJpb3MvYmlvcy0yNTZrLmJpbgpkb21haW5idWlsZGVyOiBkZXRhaWw6IHhjX2RvbV9ib290X3hl bl9pbml0OiB2ZXIgNC4xNywgY2FwcyB4ZW4tMy4wLXg4Nl82NCBodm0tMy4wLXg4Nl8zMiBodm0t My4wLXg4Nl8zMnAgaHZtLTMuMC14ODZfNjQgCmRvbWFpbmJ1aWxkZXI6IGRldGFpbDogeGNfZG9t X3BhcnNlX2ltYWdlOiBjYWxsZWQKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fZmluZF9s b2FkZXI6IHRyeWluZyBtdWx0aWJvb3QtYmluYXJ5IGxvYWRlciAuLi4gCmRvbWFpbmJ1aWxkZXI6 IGRldGFpbDogbG9hZGVyIHByb2JlIGZhaWxlZApkb21haW5idWlsZGVyOiBkZXRhaWw6IHhjX2Rv bV9maW5kX2xvYWRlcjogdHJ5aW5nIEhWTS1nZW5lcmljIGxvYWRlciAuLi4gCmRvbWFpbmJ1aWxk ZXI6IGRldGFpbDogbG9hZGVyIHByb2JlIE9LCnhjOiBkZXRhaWw6IEVMRjogcGhkcjogcGFkZHI9 MHgxMDAwMDAgbWVtc3o9MHgzNDBjMAp4YzogZGV0YWlsOiBFTEY6IG1lbW9yeTogMHgxMDAwMDAg LT4gMHgxMzQwYzAKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fY29tcGF0X2NoZWNrOiBz dXBwb3J0ZWQgZ3Vlc3QgdHlwZTogeGVuLTMuMC14ODZfNjQKZG9tYWluYnVpbGRlcjogZGV0YWls OiB4Y19kb21fY29tcGF0X2NoZWNrOiBzdXBwb3J0ZWQgZ3Vlc3QgdHlwZTogaHZtLTMuMC14ODZf MzIgPD0gbWF0Y2hlcwpkb21haW5idWlsZGVyOiBkZXRhaWw6IHhjX2RvbV9jb21wYXRfY2hlY2s6 IHN1cHBvcnRlZCBndWVzdCB0eXBlOiBodm0tMy4wLXg4Nl8zMnAKZG9tYWluYnVpbGRlcjogZGV0 YWlsOiB4Y19kb21fY29tcGF0X2NoZWNrOiBzdXBwb3J0ZWQgZ3Vlc3QgdHlwZTogaHZtLTMuMC14 ODZfNjQKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fbWVtX2luaXQ6IG1lbSA0MDg4IE1C LCBwYWdlcyAweGZmODAwIHBhZ2VzLCA0ayBlYWNoCmRvbWFpbmJ1aWxkZXI6IGRldGFpbDogeGNf ZG9tX21lbV9pbml0OiAweGZmODAwIHBhZ2VzCmRvbWFpbmJ1aWxkZXI6IGRldGFpbDogeGNfZG9t X2Jvb3RfbWVtX2luaXQ6IGNhbGxlZApkb21haW5idWlsZGVyOiBkZXRhaWw6IHJhbmdlOiBzdGFy dD0weDAgZW5kPTB4ZjAwMDAwMDAKZG9tYWluYnVpbGRlcjogZGV0YWlsOiByYW5nZTogc3RhcnQ9 MHgxMDAwMDAwMDAgZW5kPTB4MTBmODAwMDAwCnhjOiBkZXRhaWw6IFBIWVNJQ0FMIE1FTU9SWSBB TExPQ0FUSU9OOgp4YzogZGV0YWlsOiAgIDRLQiBQQUdFUzogMHgwMDAwMDAwMDAwMDAyODAwCnhj OiBkZXRhaWw6ICAgMk1CIFBBR0VTOiAweDAwMDAwMDAwMDAwMDA3ZTgKeGM6IGRldGFpbDogICAx R0IgUEFHRVM6IDB4MDAwMDAwMDAwMDAwMDAwMApkb21haW5idWlsZGVyOiBkZXRhaWw6IHhjX2Rv bV9idWlsZF9pbWFnZTogY2FsbGVkCmRvbWFpbmJ1aWxkZXI6IGRldGFpbDogeGNfZG9tX3Bmbl90 b19wdHJfcmV0Y291bnQ6IGRvbVUgbWFwcGluZzogcGZuIDB4MTAwKzB4MzUgYXQgMHg3ZDMwZTY4 ZDAwMDAKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fYWxsb2Nfc2VnbWVudDogICBrZXJu ZWwgICAgICAgOiAweDEwMDAwMCAtPiAweDEzNTAwMCAgKHBmbiAweDEwMCArIDB4MzUgcGFnZXMp CnhjOiBkZXRhaWw6IEVMRjogcGhkciAwIGF0IDB4N2QzMGU2ODliMDAwIC0+IDB4N2QzMGU2OGM1 NjQwCmRvbWFpbmJ1aWxkZXI6IGRldGFpbDogeGNfZG9tX3Bmbl90b19wdHJfcmV0Y291bnQ6IGRv bVUgbWFwcGluZzogcGZuIDB4MTM1KzB4NDAgYXQgMHg3ZDMwZTY4OTAwMDAKZG9tYWluYnVpbGRl cjogZGV0YWlsOiB4Y19kb21fYWxsb2Nfc2VnbWVudDogICBTeXN0ZW0gRmlybXdhcmUgbW9kdWxl IDogMHgxMzUwMDAgLT4gMHgxNzUwMDAgIChwZm4gMHgxMzUgKyAweDQwIHBhZ2VzKQpkb21haW5i dWlsZGVyOiBkZXRhaWw6IHhjX2RvbV9wZm5fdG9fcHRyX3JldGNvdW50OiBkb21VIG1hcHBpbmc6 IHBmbiAweDE3NSsweDEgYXQgMHg3ZDMwZTZlNGMwMDAKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4 Y19kb21fYWxsb2Nfc2VnbWVudDogICBIVk0gc3RhcnQgaW5mbyA6IDB4MTc1MDAwIC0+IDB4MTc2 MDAwICAocGZuIDB4MTc1ICsgMHgxIHBhZ2VzKQpkb21haW5idWlsZGVyOiBkZXRhaWw6IHhjX2Rv bV9idWlsZF9pbWFnZSAgOiB2aXJ0X2FsbG9jX2VuZCA6IDB4MTc2MDAwCmRvbWFpbmJ1aWxkZXI6 IGRldGFpbDogeGNfZG9tX2J1aWxkX2ltYWdlICA6IHZpcnRfcGd0YWJfZW5kIDogMHgwCmRvbWFp bmJ1aWxkZXI6IGRldGFpbDogeGNfZG9tX2Jvb3RfaW1hZ2U6IGNhbGxlZApkb21haW5idWlsZGVy OiBkZXRhaWw6IGRvbWFpbiBidWlsZGVyIG1lbW9yeSBmb290cHJpbnQKZG9tYWluYnVpbGRlcjog ZGV0YWlsOiAgICBhbGxvY2F0ZWQKZG9tYWluYnVpbGRlcjogZGV0YWlsOiAgICAgICBtYWxsb2Mg ICAgICAgICAgICAgOiA1OTI4IGJ5dGVzCmRvbWFpbmJ1aWxkZXI6IGRldGFpbDogICAgICAgYW5v biBtbWFwICAgICAgICAgIDogMCBieXRlcwpkb21haW5idWlsZGVyOiBkZXRhaWw6ICAgIG1hcHBl ZApkb21haW5idWlsZGVyOiBkZXRhaWw6ICAgICAgIGZpbGUgbW1hcCAgICAgICAgICA6IDE3MCBr Qgpkb21haW5idWlsZGVyOiBkZXRhaWw6ICAgICAgIGRvbVUgbW1hcCAgICAgICAgICA6IDQ3MiBr Qgpkb21haW5idWlsZGVyOiBkZXRhaWw6IHZjcHVfaHZtOiBjYWxsZWQKZG9tYWluYnVpbGRlcjog ZGV0YWlsOiB4Y19kb21fc2V0X2dudHRhYl9lbnRyeTogZDQgZ250WzBdIC0+IGQwIDB4ZmVmZmYK ZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fc2V0X2dudHRhYl9lbnRyeTogZDQgZ250WzFd IC0+IGQwIDB4ZmVmZmMKZG9tYWluYnVpbGRlcjogZGV0YWlsOiB4Y19kb21fcmVsZWFzZTogY2Fs bGVkCmxpYnhsOiBkZWJ1ZzogbGlieGxfZGV2aWNlLmM6NDE1OmxpYnhsX19kZXZpY2VfZGlza19z ZXRfYmFja2VuZDogRGlzayB2ZGV2PWhkYSBzcGVjLmJhY2tlbmQ9cGh5IHNwZWNpZmljYXRpb249 eGVuCmxpYnhsOiBkZWJ1ZzogbGlieGxfZXZlbnQuYzo4MTI6bGlieGxfX2V2X3hzd2F0Y2hfcmVn aXN0ZXI6IHdhdGNoIHc9MHg2MWE0Y2E2NmYyZTAgd3BhdGg9L2xvY2FsL2RvbWFpbi8wL2JhY2tl bmQvdmJkLzQvNzY4L3N0YXRlIHRva2VuPTMvMDogcmVnaXN0ZXIgc2xvdG51bT0zCmxpYnhsOiBk ZWJ1ZzogbGlieGxfZGV2aWNlLmM6NDE1OmxpYnhsX19kZXZpY2VfZGlza19zZXRfYmFja2VuZDog RGlzayB2ZGV2PWhkYyBzcGVjLmJhY2tlbmQ9cWRpc2sgc3BlY2lmaWNhdGlvbj14ZW4KbGlieGw6 IGRlYnVnOiBsaWJ4bF9saW51eC5jOjIzNDpsaWJ4bF9fZ2V0X2hvdHBsdWdfc2NyaXB0X2luZm86 IERvbWFpbiA0OmJhY2tlbmRfa2luZCAzLCBubyBuZWVkIHRvIGV4ZWN1dGUgc2NyaXB0cwpsaWJ4 bDogZGVidWc6IGxpYnhsX2RldmljZS5jOjEyMzY6ZGV2aWNlX2hvdHBsdWc6IERvbWFpbiA0Ok5v IGhvdHBsdWcgc2NyaXB0IHRvIGV4ZWN1dGUKbGlieGw6IGRlYnVnOiBsaWJ4bF9ldmVudC5jOjg2 MzpsaWJ4bF9fZXZfeHN3YXRjaF9kZXJlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNhNjcwNmYwOiBk ZXJlZ2lzdGVyIHVucmVnaXN0ZXJlZApsaWJ4bDogZGVidWc6IGxpYnhsX2NyZWF0ZS5jOjIxMjA6 ZG9fZG9tYWluX2NyZWF0ZTogYW8gMHg2MWE0Y2E2NjdmZDA6IGlucHJvZ3Jlc3M6IHBvbGxlcj0w eDYxYTRjYTY2ODA3MCwgZmxhZ3M9aQpsaWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6NzUwOndh dGNoZmRfY2FsbGJhY2s6IHdhdGNoIHc9MHg2MWE0Y2E2NmYyZTAgd3BhdGg9L2xvY2FsL2RvbWFp bi8wL2JhY2tlbmQvdmJkLzQvNzY4L3N0YXRlIHRva2VuPTMvMDogZXZlbnQgZXBhdGg9L2xvY2Fs L2RvbWFpbi8wL2JhY2tlbmQvdmJkLzQvNzY4L3N0YXRlCmxpYnhsOiBkZWJ1ZzogbGlieGxfZXZl bnQuYzoxMDUxOmRldnN0YXRlX2NhbGxiYWNrOiBiYWNrZW5kIC9sb2NhbC9kb21haW4vMC9iYWNr ZW5kL3ZiZC80Lzc2OC9zdGF0ZSB3YW50ZWQgc3RhdGUgMiBvawpsaWJ4bDogZGVidWc6IGxpYnhs X2V2ZW50LmM6ODQ5OmxpYnhsX19ldl94c3dhdGNoX2RlcmVnaXN0ZXI6IHdhdGNoIHc9MHg2MWE0 Y2E2NmYyZTAgd3BhdGg9L2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvdmJkLzQvNzY4L3N0YXRlIHRv a2VuPTMvMDogZGVyZWdpc3RlciBzbG90bnVtPTMKbGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2Uu YzoxMTUwOmRldmljZV9iYWNrZW5kX2NhbGxiYWNrOiBEb21haW4gNDpjYWxsaW5nIGRldmljZV9i YWNrZW5kX2NsZWFudXAKbGlieGw6IGRlYnVnOiBsaWJ4bF9ldmVudC5jOjg2MzpsaWJ4bF9fZXZf eHN3YXRjaF9kZXJlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNhNjZmMmUwOiBkZXJlZ2lzdGVyIHVu cmVnaXN0ZXJlZApsaWJ4bDogZGVidWc6IGxpYnhsX2xpbnV4LmM6MTk0OmxpYnhsX19ob3RwbHVn X2Rpc2s6IERvbWFpbiA0OkFyZ3MgYW5kIGVudmlyb25tZW50IHJlYWR5CmxpYnhsOiBkZWJ1Zzog bGlieGxfZGV2aWNlLmM6MTI1MTpkZXZpY2VfaG90cGx1ZzogRG9tYWluIDQ6Y2FsbGluZyBob3Rw bHVnIHNjcmlwdDogL2V0Yy94ZW4vc2NyaXB0cy9ibG9jayBhZGQKbGlieGw6IGRlYnVnOiBsaWJ4 bF9kZXZpY2UuYzoxMjUyOmRldmljZV9ob3RwbHVnOiBEb21haW4gNDpleHRyYSBhcmdzOgpsaWJ4 bDogZGVidWc6IGxpYnhsX2RldmljZS5jOjEyNjA6ZGV2aWNlX2hvdHBsdWc6IERvbWFpbiA0OmVu djoKbGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzoxMjY3OmRldmljZV9ob3RwbHVnOiBEb21h aW4gNDoJc2NyaXB0OiAvZXRjL3hlbi9zY3JpcHRzL2Jsb2NrCmxpYnhsOiBkZWJ1ZzogbGlieGxf ZGV2aWNlLmM6MTI2NzpkZXZpY2VfaG90cGx1ZzogRG9tYWluIDQ6CVhFTkJVU19UWVBFOiB2YmQK bGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzoxMjY3OmRldmljZV9ob3RwbHVnOiBEb21haW4g NDoJWEVOQlVTX1BBVEg6IGJhY2tlbmQvdmJkLzQvNzY4CmxpYnhsOiBkZWJ1ZzogbGlieGxfZGV2 aWNlLmM6MTI2NzpkZXZpY2VfaG90cGx1ZzogRG9tYWluIDQ6CVhFTkJVU19CQVNFX1BBVEg6IGJh Y2tlbmQKbGlieGw6IGRlYnVnOiBsaWJ4bF9hb3V0aWxzLmM6NTkzOmxpYnhsX19hc3luY19leGVj X3N0YXJ0OiBmb3JraW5nIHRvIGV4ZWN1dGU6IC9ldGMveGVuL3NjcmlwdHMvYmxvY2sgYWRkIAps aWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6NzIwOndhdGNoZmRfY2FsbGJhY2s6IHdhdGNoIGVw YXRoPS9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3ZiZC80Lzc2OC9zdGF0ZSB0b2tlbj0zLzA6IGVt cHR5IHNsb3QKbGlieGw6IGRlYnVnOiBsaWJ4bF9ldmVudC5jOjg2MzpsaWJ4bF9fZXZfeHN3YXRj aF9kZXJlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNhNjZmM2UwOiBkZXJlZ2lzdGVyIHVucmVnaXN0 ZXJlZApsaWJ4bDogZGVidWc6IGxpYnhsX2xpbnV4LmM6MjExOmxpYnhsX19nZXRfaG90cGx1Z19z Y3JpcHRfaW5mbzogRG9tYWluIDQ6bnVtX2V4ZWMgMSwgbm90IHJ1bm5pbmcgaG90cGx1ZyBzY3Jp cHRzCmxpYnhsOiBkZWJ1ZzogbGlieGxfZGV2aWNlLmM6MTIzNjpkZXZpY2VfaG90cGx1ZzogRG9t YWluIDQ6Tm8gaG90cGx1ZyBzY3JpcHQgdG8gZXhlY3V0ZQpsaWJ4bDogZGVidWc6IGxpYnhsX2V2 ZW50LmM6ODYzOmxpYnhsX19ldl94c3dhdGNoX2RlcmVnaXN0ZXI6IHdhdGNoIHc9MHg2MWE0Y2E2 NmYzZTA6IGRlcmVnaXN0ZXIgdW5yZWdpc3RlcmVkCmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzox Nzk6bGlieGxfX2RvbWFpbl9nZXRfZGV2aWNlX21vZGVsX3VpZDogRG9tYWluIDQ6ZG1fcmVzdHJp Y3QgZGlzYWJsZWQsIHN0YXJ0aW5nIFFFTVUgYXMgcm9vdApsaWJ4bDogZGVidWc6IGxpYnhsX2Rp c2suYzoxMjkyOmxpYnhsX19kZXZpY2VfZGlza19maW5kX2xvY2FsX3BhdGg6IERpcmVjdGx5IGFj Y2Vzc2luZyBsb2NhbCBSQVcgZGlzayAvZGV2L3ZnMC90b29sYmVsdC1yb290CmxpYnhsOiBkZWJ1 ZzogbGlieGxfZGlzay5jOjEyOTI6bGlieGxfX2RldmljZV9kaXNrX2ZpbmRfbG9jYWxfcGF0aDog RGlyZWN0bHkgYWNjZXNzaW5nIGxvY2FsIFJBVyBkaXNrIC9kL2RtZy91YnVudHUtMjQuMDQtbGl2 ZS1zZXJ2ZXItYW1kNjQuaXNvCmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk1OmxpYnhsX19z cGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6U3Bhd25pbmcgZGV2aWNlLW1vZGVsIC91c3IvbGliZXhl Yy94ZW4tcWVtdS1zeXN0ZW0taTM4NiB3aXRoIGFyZ3VtZW50czoKbGlieGw6IGRlYnVnOiBsaWJ4 bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC91c3IvbGliZXhl Yy94ZW4tcWVtdS1zeXN0ZW0taTM4NgpsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4 bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgLXhlbi1kb21pZApsaWJ4bDogZGVidWc6IGxp YnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgNApsaWJ4bDog ZGVidWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAg LW5vLXNodXRkb3duCmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9s b2NhbF9kbTogRG9tYWluIDQ6ICAtY2hhcmRldgpsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6Mjk5 NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgc29ja2V0LGlkPWxpYnhsLWNtZCxw YXRoPS92YXIvcnVuL3hlbi9xbXAtbGlieGwtNCxzZXJ2ZXI9b24sd2FpdD1vZmYKbGlieGw6IGRl YnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC1t b24KbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBE b21haW4gNDogIGNoYXJkZXY9bGlieGwtY21kLG1vZGU9Y29udHJvbApsaWJ4bDogZGVidWc6IGxp YnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgLWNoYXJkZXYK bGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21h aW4gNDogIHNvY2tldCxpZD1saWJ4ZW5zdGF0LWNtZCxwYXRoPS92YXIvcnVuL3hlbi9xbXAtbGli eGVuc3RhdC00LHNlcnZlcj1vbix3YWl0PW9mZgpsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6Mjk5 NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgLW1vbgpsaWJ4bDogZGVidWc6IGxp YnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgY2hhcmRldj1s aWJ4ZW5zdGF0LWNtZCxtb2RlPWNvbnRyb2wKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6 bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC1ub2RlZmF1bHRzCmxpYnhsOiBkZWJ1 ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICAtbm8t dXNlci1jb25maWcKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xv Y2FsX2RtOiBEb21haW4gNDogIC1uYW1lCmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk3Omxp YnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICB0b29sYmVsdApsaWJ4bDogZGVidWc6IGxp YnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgLXZuYwpsaWJ4 bDogZGVidWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0 OiAgMTI3LjAuMC4xOjEsdG89OTkKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxf X3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC1kaXNwbGF5CmxpYnhsOiBkZWJ1ZzogbGlieGxf ZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICBub25lCmxpYnhsOiBk ZWJ1ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICAt ZGV2aWNlCmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9k bTogRG9tYWluIDQ6ICBjaXJydXMtdmdhLHZnYW1lbV9tYj04CmxpYnhsOiBkZWJ1ZzogbGlieGxf ZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICAtYm9vdApsaWJ4bDog ZGVidWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAg b3JkZXI9ZGMKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2Fs X2RtOiBEb21haW4gNDogIC1zbXAKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxf X3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIDIsbWF4Y3B1cz0yCmxpYnhsOiBkZWJ1ZzogbGli eGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICAtZGV2aWNlCmxp YnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWlu IDQ6ICBydGw4MTM5LGlkPW5pYzAsbmV0ZGV2PW5ldDAsbWFjPTAwOjE2OjNlOjQzOmJkOmI1Cmxp YnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWlu IDQ6ICAtbmV0ZGV2CmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzoyOTk3OmxpYnhsX19zcGF3bl9s b2NhbF9kbTogRG9tYWluIDQ6ICB0eXBlPXRhcCxpZD1uZXQwLGlmbmFtZT12aWY0LjAtZW11LGJy PXhlbmJyMCxzY3JpcHQ9bm8sZG93bnNjcmlwdD1ubwpsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6 Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgLW1hY2hpbmUKbGlieGw6IGRl YnVnOiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIHhl bmZ2LHN1cHByZXNzLXZtZGVzYz1vbgpsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4 bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgLW0KbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5j OjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIDQwODgKbGlieGw6IGRlYnVn OiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC1kcml2 ZQpsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERv bWFpbiA0OiAgZmlsZT0vZGV2L3ZnMC90b29sYmVsdC1yb290LGlmPWlkZSxpbmRleD0wLG1lZGlh PWRpc2ssZm9ybWF0PXJhdyxjYWNoZT13cml0ZWJhY2sKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5j OjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC1kcml2ZQpsaWJ4bDogZGVi dWc6IGxpYnhsX2RtLmM6Mjk5NzpsaWJ4bF9fc3Bhd25fbG9jYWxfZG06IERvbWFpbiA0OiAgaWY9 bm9uZSxyZWFkb25seT1vbixpZD1pZGUtNTYzMixmaWxlPS9kL2RtZy91YnVudHUtMjQuMDQtbGl2 ZS1zZXJ2ZXItYW1kNjQuaXNvLGZvcm1hdD1yYXcKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjI5 OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIC1kZXZpY2UKbGlieGw6IGRlYnVn OiBsaWJ4bF9kbS5jOjI5OTc6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBEb21haW4gNDogIGlkZS1j ZCxpZD1pZGUtNTYzMixkcml2ZT1pZGUtNTYzMixidXM9aWRlLjEsdW5pdD0wCmxpYnhsOiBkZWJ1 ZzogbGlieGxfZG0uYzoyOTk5OmxpYnhsX19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6U3Bhd25p bmcgZGV2aWNlLW1vZGVsIC91c3IvbGliZXhlYy94ZW4tcWVtdS1zeXN0ZW0taTM4NiB3aXRoIGFk ZGl0aW9uYWwgZW52aXJvbm1lbnQ6CmxpYnhsOiBkZWJ1ZzogbGlieGxfZG0uYzozMDAxOmxpYnhs X19zcGF3bl9sb2NhbF9kbTogRG9tYWluIDQ6ICBYRU5fUUVNVV9DT05TT0xFX0xJTUlUPTEwNDg1 NzYKbGlieGw6IGRlYnVnOiBsaWJ4bF9kbS5jOjMwMDE6bGlieGxfX3NwYXduX2xvY2FsX2RtOiBE b21haW4gNDogIFhFTl9ET01BSU5fSUQ9NApsaWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6ODEy OmxpYnhsX19ldl94c3dhdGNoX3JlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNhNjZkYjEwIHdwYXRo PS9sb2NhbC9kb21haW4vMC9kZXZpY2UtbW9kZWwvNC9zdGF0ZSB0b2tlbj0zLzE6IHJlZ2lzdGVy IHNsb3RudW09MwpsaWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6NzUwOndhdGNoZmRfY2FsbGJh Y2s6IHdhdGNoIHc9MHg2MWE0Y2E2NmRiMTAgd3BhdGg9L2xvY2FsL2RvbWFpbi8wL2RldmljZS1t b2RlbC80L3N0YXRlIHRva2VuPTMvMTogZXZlbnQgZXBhdGg9L2xvY2FsL2RvbWFpbi8wL2Rldmlj ZS1tb2RlbC80L3N0YXRlCmxpYnhsOiBkZWJ1ZzogbGlieGxfZXhlYy5jOjQwNzpzcGF3bl93YXRj aF9ldmVudDogZG9tYWluIDQgZGV2aWNlIG1vZGVsOiBzcGF3biB3YXRjaCBwPShudWxsKQpsaWJ4 bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6ODQ5OmxpYnhsX19ldl94c3dhdGNoX2RlcmVnaXN0ZXI6 IHdhdGNoIHc9MHg2MWE0Y2E2NmRiMTAgd3BhdGg9L2xvY2FsL2RvbWFpbi8wL2RldmljZS1tb2Rl bC80L3N0YXRlIHRva2VuPTMvMTogZGVyZWdpc3RlciBzbG90bnVtPTMKbGlieGw6IGVycm9yOiBs aWJ4bF9kbS5jOjMxNTE6ZGV2aWNlX21vZGVsX3NwYXduX291dGNvbWU6IERvbWFpbiA0OmRvbWFp biA0IGRldmljZSBtb2RlbDogc3Bhd24gZmFpbGVkIChyYz0tMykKbGlieGw6IGVycm9yOiBsaWJ4 bF9kbS5jOjMzNzE6ZGV2aWNlX21vZGVsX3Bvc3Rjb25maWdfZG9uZTogRG9tYWluIDQ6UG9zdCBE TSBzdGFydHVwIGNvbmZpZ3MgZmFpbGVkLCByYz0tMwpsaWJ4bDogZGVidWc6IGxpYnhsX3FtcC5j OjE5MjA6bGlieGxfX2V2X3FtcF9kaXNwb3NlOiAgZXYgMHg2MWE0Y2E2NmRiMjgKbGlieGw6IGVy cm9yOiBsaWJ4bF9jcmVhdGUuYzoxODk2OmRvbWNyZWF0ZV9kZXZtb2RlbF9zdGFydGVkOiBEb21h aW4gNDpkZXZpY2UgbW9kZWwgZGlkIG5vdCBzdGFydDogLTMKbGlieGw6IGRlYnVnOiBsaWJ4bF9l dmVudC5jOjg2MzpsaWJ4bF9fZXZfeHN3YXRjaF9kZXJlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNh NjZmMWUwOiBkZXJlZ2lzdGVyIHVucmVnaXN0ZXJlZApsaWJ4bDogZGVidWc6IGxpYnhsX2RtLmM6 MzU4NTpsaWJ4bF9fZGVzdHJveV9kZXZpY2VfbW9kZWw6IERvbWFpbiA0OkRpZG4ndCBmaW5kIGRt IFVJRDsgZGVzdHJveWluZyBieSBwaWQKbGlieGw6IGVycm9yOiBsaWJ4bF9hb3V0aWxzLmM6NjQ2 OmxpYnhsX19raWxsX3hzX3BhdGg6IERldmljZSBNb2RlbCBhbHJlYWR5IGV4aXRlZApsaWJ4bDog ZGVidWc6IGxpYnhsX2V2ZW50LmM6ODEyOmxpYnhsX19ldl94c3dhdGNoX3JlZ2lzdGVyOiB3YXRj aCB3PTB4NjFhNGNhNjcxNDMwIHdwYXRoPS9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3ZiZC80Lzc2 OC9zdGF0ZSB0b2tlbj0zLzI6IHJlZ2lzdGVyIHNsb3RudW09MwpsaWJ4bDogZGVidWc6IGxpYnhs X2V2ZW50LmM6NzUwOndhdGNoZmRfY2FsbGJhY2s6IHdhdGNoIHc9MHg2MWE0Y2E2NzE0MzAgd3Bh dGg9L2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvdmJkLzQvNzY4L3N0YXRlIHRva2VuPTMvMjogZXZl bnQgZXBhdGg9L2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvdmJkLzQvNzY4L3N0YXRlCmxpYnhsOiBk ZWJ1ZzogbGlieGxfZXZlbnQuYzoxMDUxOmRldnN0YXRlX2NhbGxiYWNrOiBiYWNrZW5kIC9sb2Nh bC9kb21haW4vMC9iYWNrZW5kL3ZiZC80Lzc2OC9zdGF0ZSB3YW50ZWQgc3RhdGUgNiBvawpsaWJ4 bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6ODQ5OmxpYnhsX19ldl94c3dhdGNoX2RlcmVnaXN0ZXI6 IHdhdGNoIHc9MHg2MWE0Y2E2NzE0MzAgd3BhdGg9L2xvY2FsL2RvbWFpbi8wL2JhY2tlbmQvdmJk LzQvNzY4L3N0YXRlIHRva2VuPTMvMjogZGVyZWdpc3RlciBzbG90bnVtPTMKbGlieGw6IGRlYnVn OiBsaWJ4bF9kZXZpY2UuYzoxMTUwOmRldmljZV9iYWNrZW5kX2NhbGxiYWNrOiBEb21haW4gNDpj YWxsaW5nIGRldmljZV9iYWNrZW5kX2NsZWFudXAKbGlieGw6IGRlYnVnOiBsaWJ4bF9ldmVudC5j Ojg2MzpsaWJ4bF9fZXZfeHN3YXRjaF9kZXJlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNhNjcxNDMw OiBkZXJlZ2lzdGVyIHVucmVnaXN0ZXJlZApsaWJ4bDogZGVidWc6IGxpYnhsX2xpbnV4LmM6MTk0 OmxpYnhsX19ob3RwbHVnX2Rpc2s6IERvbWFpbiA0OkFyZ3MgYW5kIGVudmlyb25tZW50IHJlYWR5 CmxpYnhsOiBkZWJ1ZzogbGlieGxfZGV2aWNlLmM6MTI1MTpkZXZpY2VfaG90cGx1ZzogRG9tYWlu IDQ6Y2FsbGluZyBob3RwbHVnIHNjcmlwdDogL2V0Yy94ZW4vc2NyaXB0cy9ibG9jayByZW1vdmUK bGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzoxMjUyOmRldmljZV9ob3RwbHVnOiBEb21haW4g NDpleHRyYSBhcmdzOgpsaWJ4bDogZGVidWc6IGxpYnhsX2RldmljZS5jOjEyNjA6ZGV2aWNlX2hv dHBsdWc6IERvbWFpbiA0OmVudjoKbGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzoxMjY3OmRl dmljZV9ob3RwbHVnOiBEb21haW4gNDoJc2NyaXB0OiAvZXRjL3hlbi9zY3JpcHRzL2Jsb2NrCmxp YnhsOiBkZWJ1ZzogbGlieGxfZGV2aWNlLmM6MTI2NzpkZXZpY2VfaG90cGx1ZzogRG9tYWluIDQ6 CVhFTkJVU19UWVBFOiB2YmQKbGlieGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzoxMjY3OmRldmlj ZV9ob3RwbHVnOiBEb21haW4gNDoJWEVOQlVTX1BBVEg6IGJhY2tlbmQvdmJkLzQvNzY4CmxpYnhs OiBkZWJ1ZzogbGlieGxfZGV2aWNlLmM6MTI2NzpkZXZpY2VfaG90cGx1ZzogRG9tYWluIDQ6CVhF TkJVU19CQVNFX1BBVEg6IGJhY2tlbmQKbGlieGw6IGRlYnVnOiBsaWJ4bF9hb3V0aWxzLmM6NTkz OmxpYnhsX19hc3luY19leGVjX3N0YXJ0OiBmb3JraW5nIHRvIGV4ZWN1dGU6IC9ldGMveGVuL3Nj cmlwdHMvYmxvY2sgcmVtb3ZlIApsaWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6NzIwOndhdGNo ZmRfY2FsbGJhY2s6IHdhdGNoIGVwYXRoPS9sb2NhbC9kb21haW4vMC9iYWNrZW5kL3ZiZC80Lzc2 OC9zdGF0ZSB0b2tlbj0zLzI6IGVtcHR5IHNsb3QKbGlieGw6IGRlYnVnOiBsaWJ4bF9ldmVudC5j Ojg2MzpsaWJ4bF9fZXZfeHN3YXRjaF9kZXJlZ2lzdGVyOiB3YXRjaCB3PTB4NjFhNGNhNjcxNTMw OiBkZXJlZ2lzdGVyIHVucmVnaXN0ZXJlZApsaWJ4bDogZGVidWc6IGxpYnhsX2xpbnV4LmM6MjEx OmxpYnhsX19nZXRfaG90cGx1Z19zY3JpcHRfaW5mbzogRG9tYWluIDQ6bnVtX2V4ZWMgMSwgbm90 IHJ1bm5pbmcgaG90cGx1ZyBzY3JpcHRzCmxpYnhsOiBkZWJ1ZzogbGlieGxfZGV2aWNlLmM6MTIz NjpkZXZpY2VfaG90cGx1ZzogRG9tYWluIDQ6Tm8gaG90cGx1ZyBzY3JpcHQgdG8gZXhlY3V0ZQps aWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6ODYzOmxpYnhsX19ldl94c3dhdGNoX2RlcmVnaXN0 ZXI6IHdhdGNoIHc9MHg2MWE0Y2E2NzE1MzA6IGRlcmVnaXN0ZXIgdW5yZWdpc3RlcmVkCmxpYnhs OiBkZWJ1ZzogbGlieGxfbGludXguYzoyMzQ6bGlieGxfX2dldF9ob3RwbHVnX3NjcmlwdF9pbmZv OiBEb21haW4gNDpiYWNrZW5kX2tpbmQgMywgbm8gbmVlZCB0byBleGVjdXRlIHNjcmlwdHMKbGli eGw6IGRlYnVnOiBsaWJ4bF9kZXZpY2UuYzoxMjM2OmRldmljZV9ob3RwbHVnOiBEb21haW4gNDpO byBob3RwbHVnIHNjcmlwdCB0byBleGVjdXRlCmxpYnhsOiBkZWJ1ZzogbGlieGxfZXZlbnQuYzo4 NjM6bGlieGxfX2V2X3hzd2F0Y2hfZGVyZWdpc3Rlcjogd2F0Y2ggdz0weDYxYTRjYTY3MTEyMDog ZGVyZWdpc3RlciB1bnJlZ2lzdGVyZWQKbGlieGw6IGRlYnVnOiBsaWJ4bF9saW51eC5jOjIzNDps aWJ4bF9fZ2V0X2hvdHBsdWdfc2NyaXB0X2luZm86IERvbWFpbiA0OmJhY2tlbmRfa2luZCA2LCBu byBuZWVkIHRvIGV4ZWN1dGUgc2NyaXB0cwpsaWJ4bDogZGVidWc6IGxpYnhsX2RldmljZS5jOjEy MzY6ZGV2aWNlX2hvdHBsdWc6IERvbWFpbiA0Ok5vIGhvdHBsdWcgc2NyaXB0IHRvIGV4ZWN1dGUK bGlieGw6IGRlYnVnOiBsaWJ4bF9ldmVudC5jOjg2MzpsaWJ4bF9fZXZfeHN3YXRjaF9kZXJlZ2lz dGVyOiB3YXRjaCB3PTB4NjFhNGNhNjc2MDQwOiBkZXJlZ2lzdGVyIHVucmVnaXN0ZXJlZApsaWJ4 bDogZGVidWc6IGxpYnhsX2RvbWFpbi5jOjE1OTc6ZGV2aWNlc19kZXN0cm95X2NiOiBEb21haW4g NDpGb3JrZWQgcGlkIDE4NTMgZm9yIGRlc3Ryb3kgb2YgZG9tYWluCmxpYnhsOiBkZWJ1ZzogbGli eGxfZXZlbnQuYzoyMDY3OmxpYnhsX19hb19jb21wbGV0ZTogYW8gMHg2MWE0Y2E2NjdmZDA6IGNv bXBsZXRlLCByYz0tMwpsaWJ4bDogZGVidWc6IGxpYnhsX2V2ZW50LmM6MjAzNjpsaWJ4bF9fYW9f X2Rlc3Ryb3k6IGFvIDB4NjFhNGNhNjY3ZmQwOiBkZXN0cm95CmxpYnhsOiBkZWJ1ZzogbGlieGxf ZG9tYWluLmM6MTA0NjpsaWJ4bF9kb21haW5fZGVzdHJveTogRG9tYWluIDQ6YW8gMHg2MWE0Y2E2 NzQ2MzA6IGNyZWF0ZTogaG93PShuaWwpIGNhbGxiYWNrPShuaWwpIHBvbGxlcj0weDYxYTRjYTY2 ODA3MApsaWJ4bDogZXJyb3I6IGxpYnhsX2RvbWFpbi5jOjExODM6bGlieGxfX2Rlc3Ryb3lfZG9t aWQ6IERvbWFpbiA0Ok5vbi1leGlzdGFudCBkb21haW4KbGlieGw6IGVycm9yOiBsaWJ4bF9kb21h aW4uYzoxMTM3OmRvbWFpbl9kZXN0cm95X2NhbGxiYWNrOiBEb21haW4gNDpVbmFibGUgdG8gZGVz dHJveSBndWVzdApsaWJ4bDogZXJyb3I6IGxpYnhsX2RvbWFpbi5jOjEwNjQ6ZG9tYWluX2Rlc3Ry b3lfY2I6IERvbWFpbiA0OkRlc3RydWN0aW9uIG9mIGRvbWFpbiBmYWlsZWQKbGlieGw6IGRlYnVn OiBsaWJ4bF9ldmVudC5jOjIwNjc6bGlieGxfX2FvX2NvbXBsZXRlOiBhbyAweDYxYTRjYTY3NDYz MDogY29tcGxldGUsIHJjPS0yMQpsaWJ4bDogZGVidWc6IGxpYnhsX2RvbWFpbi5jOjEwNTU6bGli eGxfZG9tYWluX2Rlc3Ryb3k6IERvbWFpbiA0OmFvIDB4NjFhNGNhNjc0NjMwOiBpbnByb2dyZXNz OiBwb2xsZXI9MHg2MWE0Y2E2NjgwNzAsIGZsYWdzPWljCmxpYnhsOiBkZWJ1ZzogbGlieGxfZXZl bnQuYzoyMDM2OmxpYnhsX19hb19fZGVzdHJveTogYW8gMHg2MWE0Y2E2NzQ2MzA6IGRlc3Ryb3kK eGVuY2FsbDpidWZmZXI6IGRlYnVnOiB0b3RhbCBhbGxvY2F0aW9uczoxNzM4IHRvdGFsIHJlbGVh c2VzOjE3MzgKeGVuY2FsbDpidWZmZXI6IGRlYnVnOiBjdXJyZW50IGFsbG9jYXRpb25zOjAgbWF4 aW11bSBhbGxvY2F0aW9uczozCnhlbmNhbGw6YnVmZmVyOiBkZWJ1ZzogY2FjaGUgY3VycmVudCBz aXplOjMKeGVuY2FsbDpidWZmZXI6IGRlYnVnOiBjYWNoZSBoaXRzOjE3MTUgbWlzc2VzOjMgdG9v YmlnOjIwCnhlbmNhbGw6YnVmZmVyOiBkZWJ1ZzogdG90YWwgYWxsb2NhdGlvbnM6MCB0b3RhbCBy ZWxlYXNlczowCnhlbmNhbGw6YnVmZmVyOiBkZWJ1ZzogY3VycmVudCBhbGxvY2F0aW9uczowIG1h eGltdW0gYWxsb2NhdGlvbnM6MAp4ZW5jYWxsOmJ1ZmZlcjogZGVidWc6IGNhY2hlIGN1cnJlbnQg c2l6ZTowCnhlbmNhbGw6YnVmZmVyOiBkZWJ1ZzogY2FjaGUgaGl0czowIG1pc3NlczowIHRvb2Jp ZzowCg== --0000000000008479d706286d66d2-- From xen-users-bounces@lists.xenproject.org Wed Dec 04 22:25:27 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Wed, 04 Dec 2024 22:25:27 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848755.1263572 (Exim 4.92) (envelope-from ) id 1tIxnT-0000aL-ED; Wed, 04 Dec 2024 22:24:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848755.1263572; Wed, 04 Dec 2024 22:24:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIxnT-0000aE-B8; Wed, 04 Dec 2024 22:24:39 +0000 Received: by outflank-mailman (input) for mailman id 848755; Wed, 04 Dec 2024 22:24:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tIxnR-0000a8-Pb for xen-users@lists.xenproject.org; Wed, 04 Dec 2024 22:24:37 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 899357a3-b28e-11ef-99a3-01e77a169b0f; Wed, 04 Dec 2024 23:24:34 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 899357a3-b28e-11ef-99a3-01e77a169b0f Date: Wed, 4 Dec 2024 22:21:38 +0000 From: Mike To: xen-users@lists.xenproject.org Subject: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: > One of the first commands that I try to execute in one > domU, which causes activity in the other domU, triggers a reboot. > > So I pinned each of the domUs' vCPUs: one with `cpus = "4-5"` and the other > with `cpus = "all,^0-5"`. Also reduced `vcpus` and `maxvcpus` in the latter, > to avoid oversubscription. Testing... So what "solved" the issue, or at least seems to avoid it for now, is increasing the `memory` and `maxmem` of the first domU to `4096` and the `vcpus` and `maxvcpus` to `3`. I probably didn't even need to increase the vCPUs; I think it's a memory constraint issue. The first domU is the sole control node for a Kubernetes cluster. It also has 2GB of swap available to it, but I'm not sure that k8s will use it, since it seems to avoid swap for sensitive data. The minimum suggested for k8s is two CPUs and 2GB of memory, which is what I initially chose. The second domU is the sole worker node for the cluster. The command that I ran in it that triggered the reboot was `kubectl delete -f` of a Deployment that was already running from an `apply`. A `free -m` on the control node domU outputs: --- total used free shared buff/cache available Mem: 3913 1391 856 2 1894 2521 Swap: 2047 0 2047 --- This seems likely to be a Xen bug. An attempt to overuse memory from inside a domU shouldn't reboot the hardware. From xen-users-bounces@lists.xenproject.org Thu Dec 05 05:19:16 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Thu, 05 Dec 2024 05:19:16 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848815.1263608 (Exim 4.92) (envelope-from ) id 1tJ4Fx-0005O8-Tr; Thu, 05 Dec 2024 05:18:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848815.1263608; Thu, 05 Dec 2024 05:18:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJ4Fx-0005O1-Qi; Thu, 05 Dec 2024 05:18:29 +0000 Received: by outflank-mailman (input) for mailman id 848815; Thu, 05 Dec 2024 05:18:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJ4Fv-0005MM-HE for xen-users@lists.xenproject.org; Thu, 05 Dec 2024 05:18:28 +0000 Received: from gw3.antarean.org (gw3.antarean.org [84.247.13.64]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 59d654e6-b2c8-11ef-a0d5-8be0dac302b0; Thu, 05 Dec 2024 06:18:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by gw3.antarean.org (Postfix) with ESMTP id 4Y3jND12FjzNsgn for ; Thu, 05 Dec 2024 06:18:36 +0100 (CET) Received: from gw3.antarean.org ([127.0.0.1]) by localhost (gw3.antarean.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v8aAUOJpyWDQ for ; Thu, 5 Dec 2024 06:18:35 +0100 (CET) Received: from mailstore1.adm.antarean.org (localhost [127.0.0.1]) by gw3.antarean.org (Postfix) with ESMTP id 4Y3jNC6FKYzNmyF for ; Thu, 05 Dec 2024 06:18:35 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mailstore1.adm.antarean.org (Postfix) with ESMTP id 4Y3jMz3pdCz1G for ; Thu, 05 Dec 2024 05:18:23 +0000 (UTC) Received: from mailstore1.adm.antarean.org ([127.0.0.1]) by localhost (mailstore1.adm.antarean.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ddxUAyRWyMxk for ; Thu, 5 Dec 2024 05:18:23 +0000 (UTC) Received: from c52aeb976e0c (web2.adm.antarean.org [10.55.16.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mailstore1.adm.antarean.org (Postfix) with ESMTPSA id 4Y3jMz0qw5z17 for ; Thu, 05 Dec 2024 05:18:23 +0000 (UTC) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 59d654e6-b2c8-11ef-a0d5-8be0dac302b0 X-Virus-Scanned: amavisd-new at antarean.org X-Virus-Scanned: amavisd-new at antarean.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=antarean.org; s=default; t=1733375903; bh=LaNCVTjxAk6cBTkM2w6DFyHX2nuMgLU4tuL7XqDyr/c=; h=From:In-Reply-To:References:List-Id:To:Subject:Date; b=je7cZ8F+vjZ0sr7ptc3XndEDRYokvZklY2wocLAuKf+PttIwuH/uY/+wtD6mLbDw6 XQuSbpZRfqvIWHmxVzpTWm8nLauNwXYwd/t+ojpLy3TCpfRp7cd7oy8+rqe1v5DMf4 VVZ66lFNVT00r3hWV9nnGGPHjJaEPPsV97GfURmY= User-Agent: EGroupware API 23.1.005 From: Joost Roeleveld X-Priority: 3 X-Mailer: EGroupware-Mail In-Reply-To: References: To: xen-users@lists.xenproject.org Subject: Re: domU memory exceeded domU memory exceeded Re: domU memory exceeded domU memory exceeded =?utf-8?b?PT89Pg==?= spontaneous reboots Message-ID: <20241205051823.EGroupware.jOxFWUkEGKzCReVrvrVF2qm@_> Date: Thu, 05 Dec 2024 05:18:23 +0000 Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes MIME-Version: 1.0 ----------------------- Original message ----------------------- From: Mike To: xen-users@lists.xenproject.org Date: Wed, 04 Dec 2024 23:21:38 +0100 ---------------------------------------------------------------- >> One of the first commands that I try to execute in one >> domU, which causes activity in the other domU, triggers a reboot. >> >> So I pinned each of the domUs' vCPUs: one with `cpus = "4-5"` and the other >> with `cpus = "all,^0-5"`. Also reduced `vcpus` and `maxvcpus` in the latter, >> to avoid oversubscription. Testing... I overcommit my CPUs probably more then I should and never had a reboot caused by this. One of my systems currently has 5 times as many vCPUs assigned as physical cores. (This one is scheduled to be replaced) Apart from being a bit slow, it is rock solid. > So what "solved" the issue, or at least seems to avoid it for now, > is increasing > the `memory` and `maxmem` of the first domU to `4096` and the `vcpus` and > `maxvcpus` to `3`. I probably didn't even need to increase the vCPUs; I think > it's a memory constraint issue. How do you overcommit memory? In default config, it won't let me start a domU if the memory is filled. Except I do fix the dom0 to never allow the memory to be removed using: "dom0_mem=4096M,max:4096M" in the xen-boot options. If you do allow the memory to be removed, you risk the OOM killer to remove something important. I would recommend monitoring the dom0 to see what happens there. > > This seems likely to be a Xen bug. An attempt to overuse memory from inside > a domU shouldn't reboot the hardware. It doesn't. The domU will crash, but won't kill the hardware. Unless you configured the domU to be privileged enough. -- Joost From xen-users-bounces@lists.xenproject.org Thu Dec 05 07:59:07 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Thu, 05 Dec 2024 07:59:07 +0000 Received: from list by lists.xenproject.org with outflank-mailman.848948.1263704 (Exim 4.92) (envelope-from ) id 1tJ6kg-0008QD-Bx; Thu, 05 Dec 2024 07:58:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 848948.1263704; Thu, 05 Dec 2024 07:58:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJ6kg-0008Q6-8O; Thu, 05 Dec 2024 07:58:22 +0000 Received: by outflank-mailman (input) for mailman id 848948; Thu, 05 Dec 2024 07:58:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJ6ke-0008Pw-VR for xen-users@lists.xenproject.org; Thu, 05 Dec 2024 07:58:21 +0000 Received: from mo4-p00-ob.smtp.rzone.de (mo4-p00-ob.smtp.rzone.de [81.169.146.217]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b09ae60e-b2de-11ef-a0d5-8be0dac302b0; Thu, 05 Dec 2024 08:58:19 +0100 (CET) Received: from mail.onlineschubla.de by smtp.strato.de (RZmta 51.2.11 DYNA|AUTH) with ESMTPSA id ed495e0B57wHFFg (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate) for ; Thu, 5 Dec 2024 08:58:17 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.onlineschubla.de (Postfix) with ESMTP id A3D8B2038A for ; Thu, 5 Dec 2024 08:58:17 +0100 (CET) Received: from mail.onlineschubla.de ([127.0.0.1]) by localhost (mail.onlineschubla.de [127.0.0.1]) (amavis, port 10024) with ESMTP id Y6a3wPXGzGPi for ; Thu, 5 Dec 2024 08:58:17 +0100 (CET) Received: from [10.0.0.105] (unknown [10.0.0.105]) by mail.onlineschubla.de (Postfix) with ESMTPA id 1FDF320131 for ; Thu, 5 Dec 2024 08:58:17 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: b09ae60e-b2de-11ef-a0d5-8be0dac302b0 ARC-Seal: i=1; a=rsa-sha256; t=1733385498; cv=none; d=strato.com; s=strato-dkim-0002; b=aQWurY99N3oHvsoUvP7j8XFiWrizPIQpyzLJxJwbdJDcY6t+zJUpo4sP6vF2Zmd6dk rdyZ/5Suv/Yni/02ppJr3ASceHv5ld7TYaeGnyyjyEHf6PmVAI/p01VCGW5MEa/qLbU+ wFNTajpar7byd9+CSgD6ofeVbnkJFAU81fYel4NhKvq1RUpx3LvO5pBZnE+gPZ5tKQAo wvL/U0Bq8HJsn9NqU5ze5/cxqwCvdS089jQNDIepeuac6wpPJ8LbQpd8PUI90IDtp4Cf 7zVyyVCCZBXDfLcs9kDJiuNGL304/RoFlHg6btPxrTQLgG+WKSX9+LalusfuM5T6ROGp sUhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1733385498; s=strato-dkim-0002; d=strato.com; h=In-Reply-To:From:References:To:Subject:Date:Message-ID:Cc:Date:From: Subject:Sender; bh=7g5AlEIZup9aKpjZHbF5MR/SCcnUOhkP0JV+AzmgdY8=; b=b74e8HZgsiXLZmifiPUuUamwk5CwsxmDDaOjMfxe2v71bAZVMehlTqU3ezNPumw5so gck/xx6z5CQL7VYXoeeUW2d/VPdHVqeMtv4zpQNNq89m4QtpSx/3hL7Oa3Rv/AbOgQXH 7o+oDvP9C/V9NabbOP1ucM4e/soX+ws31MyrUJb+63i9KjVjywtrUzMkbaixx8BKAvBr BSARtP6Wv/fxzdnnNhmLvtJ4wPQXglQmnhdNe46j+OuJhADSJT0qvAvyr8wyR3i/HcH0 7mfi+WGQWp7WbFxu7S6rBjYvW+tGHJ9xvLVeWSlndNdxlkXH1hUHcgfNYyRqvWgZjs31 ttiQ== ARC-Authentication-Results: i=1; strato.com; arc=none; dkim=none X-RZG-CLASS-ID: mo00 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1733385498; s=strato-dkim-0002; d=onlineschubla.de; h=In-Reply-To:From:References:To:Subject:Date:Message-ID:Cc:Date:From: Subject:Sender; bh=7g5AlEIZup9aKpjZHbF5MR/SCcnUOhkP0JV+AzmgdY8=; b=iR5nubGuIogirb4BaZOQugEXwf8a4+di2Q1DSDqYHJC2kCzhOWhKiifX01m0wT04qn hwvNZR3P7OkTBI2NsvpkeRIvzmds0pc8WVXJ7CtEpjht/iMZ5SCUomudTuXhorCjQY/t 5kyOzFh9q7LjpvoQH9DvWsbtm7sXf2scwRuq/LiVnCWn+SLa6yOI8XfGir/w5xw/tEBK EdjAaC3r1BFHONDpMB0rQkExYYFfMEfb+ZsEgb61TIqyNlmT4D6uhmuSQprCbvnPS78Z j6aE65jAooFs6RblA6inyTp0sMO7PhmQIQipu0Mz/AbT/mNhOYP+lL2fyGZc/DA2PN69 Xflw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1733385498; s=strato-dkim-0003; d=onlineschubla.de; h=In-Reply-To:From:References:To:Subject:Date:Message-ID:Cc:Date:From: Subject:Sender; bh=7g5AlEIZup9aKpjZHbF5MR/SCcnUOhkP0JV+AzmgdY8=; b=A9c47cCTNGyoWoUL2Qpp4IVnNmvk7kpNvs5R6zxu0fQSjF54Io6pRtoKwZJURyFcKX AX3IPR1nubZ16M4laLCA== X-RZG-AUTH: ":PG0ReWCndfO3rCSML4AvNaDxJ7WJyilEI/NMX3IPpS4dskMFCImJEP1iWJS7B4nsYw==" X-Virus-Scanned: Debian amavis at onlineschubla.de Message-ID: Date: Thu, 5 Dec 2024 08:58:10 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: CPU oversubscription =?=> spontaneous reboots To: xen-users@lists.xenproject.org References: Content-Language: en-US From: Paul Leiber In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Mike, Am 03.12.2024 um 23:27 schrieb Mike: >> I added >> `dom0_vcpus_pin` to the boot line and `cpus = "all,^0-3"` to the domU configs > > That didn't solve it. One of the first commands that I try to execute in one > domU, which causes activity in the other domU, triggers a reboot. As if the > change increased the likeliness of the event. Could it be possible that it's not the activity on the DomU that is triggering the reboot, but rather network activity between two DomUs? If the cause were activity by itself on one DomU (or both DomUs), then it should be possible to trigger the reboot by causing activity on the DomU itself (or on both DomUs), without network activity. What CPU architecture is your system based on? Best regards, Paul From xen-users-bounces@lists.xenproject.org Thu Dec 05 15:18:46 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Thu, 05 Dec 2024 15:18:46 +0000 Received: from list by lists.xenproject.org with outflank-mailman.849094.1263790 (Exim 4.92) (envelope-from ) id 1tJDcA-0000RF-6I; Thu, 05 Dec 2024 15:18:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 849094.1263790; Thu, 05 Dec 2024 15:18:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJDcA-0000R8-2K; Thu, 05 Dec 2024 15:18:02 +0000 Received: by outflank-mailman (input) for mailman id 849094; Thu, 05 Dec 2024 15:18:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJDc8-0000R2-Cc for xen-users@lists.xenproject.org; Thu, 05 Dec 2024 15:18:00 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 1ae58a68-b31c-11ef-a0d5-8be0dac302b0; Thu, 05 Dec 2024 16:17:57 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 1ae58a68-b31c-11ef-a0d5-8be0dac302b0 Date: Thu, 5 Dec 2024 15:15:10 +0000 From: Mike To: Joost Roeleveld Cc: xen-users@lists.xenproject.org Subject: Re: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: <20241205051823.EGroupware.jOxFWUkEGKzCReVrvrVF2qm@_> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20241205051823.EGroupware.jOxFWUkEGKzCReVrvrVF2qm@_> Joost Roeleveld wrote: > How do you overcommit memory? I don't. I took the memory from the other domU. From xen-users-bounces@lists.xenproject.org Thu Dec 05 15:26:51 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Thu, 05 Dec 2024 15:26:51 +0000 Received: from list by lists.xenproject.org with outflank-mailman.849124.1263806 (Exim 4.92) (envelope-from ) id 1tJDkX-0002VO-15; Thu, 05 Dec 2024 15:26:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 849124.1263806; Thu, 05 Dec 2024 15:26:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJDkW-0002VH-U8; Thu, 05 Dec 2024 15:26:40 +0000 Received: by outflank-mailman (input) for mailman id 849124; Thu, 05 Dec 2024 15:26:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJDkV-0002Un-M3 for xen-users@lists.xenproject.org; Thu, 05 Dec 2024 15:26:39 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4ca5856f-b31d-11ef-99a3-01e77a169b0f; Thu, 05 Dec 2024 16:26:37 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 4ca5856f-b31d-11ef-99a3-01e77a169b0f Date: Thu, 5 Dec 2024 15:23:22 +0000 From: Mike To: Paul Leiber Cc: xen-users@lists.xenproject.org Subject: Re: CPU oversubscription =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Paul Leiber wrote: > Could it be possible that it's not the activity on the DomU that is > triggering the reboot, but rather network activity between two DomUs? Sure, that's possible. The domUs are a k8s control and worker node, respectively, so they need to communicate with each other when I issue the `kubectl delete` that trigger it. But I resolved the issue (for now) by increasing the control node's admittedly tight memory. So that doesn't point to a network issue in my mind. > If the > cause were activity by itself on one DomU (or both DomUs), then it should be > possible to trigger the reboot by causing activity on the DomU itself (or on > both DomUs), without network activity. Because it's a k8s action, there's no way to do this action on one domU alone. The command communicates to the control node, then the control node will tell the worker node what to do. > What CPU architecture is your system based on? amd64 Intel Core i9-14900T From xen-users-bounces@lists.xenproject.org Thu Dec 05 21:17:58 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Thu, 05 Dec 2024 21:17:58 +0000 Received: from list by lists.xenproject.org with outflank-mailman.849316.1263939 (Exim 4.92) (envelope-from ) id 1tJJDu-0000By-EX; Thu, 05 Dec 2024 21:17:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 849316.1263939; Thu, 05 Dec 2024 21:17:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJJDu-0000Br-BT; Thu, 05 Dec 2024 21:17:22 +0000 Received: by outflank-mailman (input) for mailman id 849316; Thu, 05 Dec 2024 21:17:21 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJJDt-0000Bl-CU for xen-users@lists.xenproject.org; Thu, 05 Dec 2024 21:17:21 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4e03f6b5-b34e-11ef-99a3-01e77a169b0f; Thu, 05 Dec 2024 22:17:18 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:8ac4:0:0:0:0:f7]) by mailhost.m5p.com (8.18.1/8.17.1) with ESMTPS id 4B5LGrfS082718 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 5 Dec 2024 16:16:58 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.18.1/8.15.2/Submit) id 4B5LGqjv082717; Thu, 5 Dec 2024 13:16:52 -0800 (PST) (envelope-from ehem) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 4e03f6b5-b34e-11ef-99a3-01e77a169b0f Date: Thu, 5 Dec 2024 13:16:52 -0800 From: Elliott Mitchell To: Mike Cc: Paul Leiber , xen-users@lists.xenproject.org Subject: Re: CPU oversubscription =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=0.3 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on mattapan.m5p.com On Wed, Dec 04, 2024 at 10:21:38PM +0000, Mike wrote: > > The second domU is the sole worker node for the cluster. The command that I > ran in it that triggered the reboot was `kubectl delete -f` of a Deployment > that was already running from an `apply`. Okay, do you have a full list of what this command does? Might it cause a crucial Xen domain to panic (domain 0) and this in turn cause Xen to panic? On Thu, Dec 05, 2024 at 03:15:10PM +0000, Mike wrote: > Joost Roeleveld wrote: > > How do you overcommit memory? > > I don't. I took the memory from the other domU. How much free memory does Xen have? Try running `xl info`, what does the "free_memory" line say? Might be 0 if Xen is ballooning memory from domain 0 to handle allocations. If ballooning memory from domain 0 has been disabled this should stay above 50 so Xen can allocate memory to handle activity. On Thu, Dec 05, 2024 at 03:23:22PM +0000, Mike wrote: > Paul Leiber wrote: > > Could it be possible that it's not the activity on the DomU that is > > triggering the reboot, but rather network activity between two DomUs? > > Sure, that's possible. The domUs are a k8s control and worker node, > respectively, so they need to communicate with each other when I issue the > `kubectl delete` that trigger it. > > But I resolved the issue (for now) by increasing the control node's > admittedly tight memory. So that doesn't point to a network issue in my > mind. Is either of these also domain 0? Domain 0 exhausting its free memory and panicing might cause the issue you're describing. > > What CPU architecture is your system based on? > > amd64 > Intel Core i9-14900T Apparently there is a major issue with 14900K processors. I've been reading mentions of other Intel 13xxx and 14xxx chips reputedly having failures at a lower rates. Right now there could still be configuration issues, but I would keep an eye out for hardware failure. -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445 From xen-users-bounces@lists.xenproject.org Fri Dec 06 06:24:06 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Fri, 06 Dec 2024 06:24:06 +0000 Received: from list by lists.xenproject.org with outflank-mailman.849848.1264365 (Exim 4.92) (envelope-from ) id 1tJRjb-0004it-P2; Fri, 06 Dec 2024 06:22:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 849848.1264365; Fri, 06 Dec 2024 06:22:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJRjb-0004im-M2; Fri, 06 Dec 2024 06:22:39 +0000 Received: by outflank-mailman (input) for mailman id 849848; Fri, 06 Dec 2024 06:22:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tJRja-0004ig-53 for xen-users@lists.xenproject.org; Fri, 06 Dec 2024 06:22:38 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 78d819b6-b39a-11ef-a0d5-8be0dac302b0; Fri, 06 Dec 2024 07:22:36 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 78d819b6-b39a-11ef-a0d5-8be0dac302b0 Date: Fri, 6 Dec 2024 06:18:09 +0000 From: Mike To: Elliott Mitchell Cc: Paul Leiber , xen-users@lists.xenproject.org Subject: Re: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Elliott Mitchell wrote: > > The command that I > > ran in it that triggered the reboot was `kubectl delete -f` of a Deployment > > that was already running from an `apply`. > > Okay, do you have a full list of what this command does? I'm sort of counting on people to know what Kubernetes does. I know that the command will contact the k8s API Server on the control node, then that will cause some cascade of communication among the k8s services running on the control node, eventually leading the API Server to contact the worker node's Kubelet to delete the objects. > Might it cause a crucial Xen domain to panic (domain 0) and this in turn > cause Xen to panic? The only way that I could see the dom0 involved is maybe via the network stack? The communication between the domUs is going through the xenbr0, of course. > How much free memory does Xen have? 265. It was sthg like 313 (= 265 + 2048 - 2000) before I moved more memory to the control node. > Might be 0 if Xen is ballooning memory from domain 0 to handle > allocations. If ballooning memory from domain 0 has been disabled this > should stay above 50 so Xen can allocate memory to handle activity. No ballooning: dom0_mem=4G;max:4G dom0_max_vcpus=4 dom0_vcpus_pin xpti=dom0=false,domu=true no-real-mode edd=off > > The domUs are a k8s control and worker node, > > Is either of these also domain 0? Domain 0 exhausting its free memory > and panicing might cause the issue you're describing. No, they are domUs. > > Intel Core i9-14900T > > Apparently there is a major issue with 14900K processors. I've been > reading mentions of other Intel 13xxx and 14xxx chips reputedly having > failures at a lower rates. The K and KS processors are designed to allow them to run at high speed, leading to temps of 90-100C that destroy the silicon. The T series are low power. Stress testing never went above 60C core temps in my configuration. > Right now there could still be configuration issues, but I would keep an > eye out for hardware failure. It's been running since June or July. I'm pretty confident that the hardware has been shaken down. The BIOS upgrade is the only thing I have my eye on. From xen-users-bounces@lists.xenproject.org Sat Dec 07 20:17:12 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Sat, 07 Dec 2024 20:17:12 +0000 Received: from list by lists.xenproject.org with outflank-mailman.850798.1265102 (Exim 4.92) (envelope-from ) id 1tK1E1-0004h3-Gu; Sat, 07 Dec 2024 20:16:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 850798.1265102; Sat, 07 Dec 2024 20:16:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tK1E1-0004gw-CY; Sat, 07 Dec 2024 20:16:25 +0000 Received: by outflank-mailman (input) for mailman id 850798; Sat, 07 Dec 2024 20:16:23 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tK1Dz-0004gq-LY for xen-users@lists.xenproject.org; Sat, 07 Dec 2024 20:16:23 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1d708f9a-b4d8-11ef-99a3-01e77a169b0f; Sat, 07 Dec 2024 21:16:19 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7]) by mailhost.m5p.com (8.18.1/8.17.1) with ESMTPS id 4B7KFkPM095931 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 7 Dec 2024 15:15:52 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.18.1/8.15.2/Submit) id 4B7KFjwo095930; Sat, 7 Dec 2024 12:15:45 -0800 (PST) (envelope-from ehem) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 1d708f9a-b4d8-11ef-99a3-01e77a169b0f Date: Sat, 7 Dec 2024 12:15:45 -0800 From: Elliott Mitchell To: Mike Cc: Paul Leiber , xen-users@lists.xenproject.org Subject: Re: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=0.3 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on mattapan.m5p.com On Fri, Dec 06, 2024 at 06:18:09AM +0000, Mike wrote: > Elliott Mitchell wrote: > > > The command that I > > > ran in it that triggered the reboot was `kubectl delete -f` of a Deployment > > > that was already running from an `apply`. > > > > Okay, do you have a full list of what this command does? > > I'm sort of counting on people to know what Kubernetes does. I know that the > command will contact the k8s API Server on the control node, then that will > cause some cascade of communication among the k8s services running on the > control node, eventually leading the API Server to contact the worker node's > Kubelet to delete the objects. Notice how this mailing list is xen-users@lists.xenproject.org? It is reasonable to think people here might have surface familiarity with Kubernetes, but you're going to need to fill in many pieces for us. > > Might it cause a crucial Xen domain to panic (domain 0) and this in turn > > cause Xen to panic? > > The only way that I could see the dom0 involved is maybe via the network > stack? The communication between the domUs is going through the xenbr0, of > course. Not knowing any of the details of how you've got things setup, do you have any pieces of Kubernetes in domain 0? These exhausting memory would cause problems. > > How much free memory does Xen have? > > 265. It was sthg like 313 (= 265 + 2048 - 2000) before I moved more memory > to the control node. Did you in fact look at the output of `xl info` (this needs to be run in domain 0)? Are you instead guessing based on what you think your computer's setup is? If you're not looking at the output of `xl info` (or `xl info | grep -e^free_memory`), then you're guessing and you could be wrong. Xen needs some memory to keep track of memory/addresses which are assigned to particular domains. Mapping pages between domains (for virtual network/block devices) does require Xen to use some memory. The estimate from Qubes OS is to start worrying if Xen's free memory drops below 50MB. If Xen's free memory is <20MB then the "network" activity causing exhaustion of Xen's memory seems plausible. If below 10MB then this seems likely. If listed as "0" then this is near-certain as the immediate cause. > > Might be 0 if Xen is ballooning memory from domain 0 to handle > > allocations. If ballooning memory from domain 0 has been disabled this > > should stay above 50 so Xen can allocate memory to handle activity. > > No ballooning: > dom0_mem=4G;max:4G dom0_max_vcpus=4 dom0_vcpus_pin xpti=dom0=false,domu=true no-real-mode edd=off Okay, but 4GB for domain 0 strikes me as an odd value. If this computer is primarily a server then 1GB should be plenty. Needing 4GB would suggest you've got way too much going on for a hypervisor control VM. Is this computer primarily your desktop and you're using Xen to generate VMs to experiment with Kubernetes? If this computer is primarily your desktop then 4GB of memory and 4 vCPUs is rather low. Many Linux desktop environments will really want 8GB and can readily benefit from 8-12 processors (lighter weight desktop environments don't need that much). > > > The domUs are a k8s control and worker node, > > > > Is either of these also domain 0? Domain 0 exhausting its free memory > > and panicing might cause the issue you're describing. > > No, they are domUs. I'm starting to suspect this computer has a total of 8GB of memory? All desktop processors support dual-channel memory. If you've only got 8GB then you're losing performance due to limited memory bandwidth. This though is assuming a desktop computer. The other trick is you *will* find you want more memory. Rule of thumb is the moment you have a VM machine, it will start eating your network. You will start finding every task on your network looks like it can be turned into a VM. There is a reason server systems have been claiming support for 2TB of memory/socket for a while. My personal VM server is enveloping my router (just needed 802.11 plus dual ethernet interfaces). I'm a bit scared since it looks hungry and seems to be eyeing my desktop. -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445 From xen-users-bounces@lists.xenproject.org Sun Dec 08 02:08:45 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Sun, 08 Dec 2024 02:08:45 +0000 Received: from list by lists.xenproject.org with outflank-mailman.850841.1265118 (Exim 4.92) (envelope-from ) id 1tK6i9-0001Oz-2I; Sun, 08 Dec 2024 02:07:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 850841.1265118; Sun, 08 Dec 2024 02:07:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tK6i8-0001Or-U6; Sun, 08 Dec 2024 02:07:52 +0000 Received: by outflank-mailman (input) for mailman id 850841; Sun, 08 Dec 2024 02:07:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tK6i7-0001Ol-Kd for xen-users@lists.xenproject.org; Sun, 08 Dec 2024 02:07:51 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 35a04031-b509-11ef-a0d5-8be0dac302b0; Sun, 08 Dec 2024 03:07:49 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 35a04031-b509-11ef-a0d5-8be0dac302b0 Date: Sun, 8 Dec 2024 02:03:58 +0000 From: Mike To: Elliott Mitchell Cc: Paul Leiber , xen-users@lists.xenproject.org Subject: Re: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Elliott Mitchell wrote: > It is > reasonable to think people here might have surface familiarity with > Kubernetes, but you're going to need to fill in many pieces for us. Yes, I know. > do you have any pieces of Kubernetes in domain 0? No. > Did you in fact look at the output of `xl info` Yes. > Are you instead guessing based on what you think your > computer's setup is? "313" is a guess based on what I recall changing, since I don't have that configuration anymore. > > dom0_mem=4G;max:4G dom0_max_vcpus=4 dom0_vcpus_pin xpti=dom0=false,domu=true no-real-mode edd=off > > Okay, but 4GB for domain 0 strikes me as an odd value. If this computer > is primarily a server then 1GB should be plenty. OK. But that's scaled back from the default, all 64GB of the memory and ballooning. I can afford it for now. I can also easily add 64GB to the machine. > Is > this computer primarily your desktop and you're using Xen to generate VMs > to experiment with Kubernetes? It's a personal server, no GUI. I'm migrating my services to Kubernetes. > I'm starting to suspect this computer has a total of 8GB of memory? 32GB x 2 DDR5-5600. I'll link you the full specs offline. > the moment you have a VM machine, it will start eating your network. I don't expect much in that regard. I'll probably have a few dozen Kubernetes pods (VMs), and Calico (a Kubernetes network plugin) handles all the inter-node (inter-domU) traffic within the domU. But there will be communicaton between every pod and the control node (the other domU), and that would pass through the xenbr0. FYI, I've been using Xen for, idk, 15-20 years? It's mostly been set-it- and-forget-it. From xen-users-bounces@lists.xenproject.org Sun Dec 08 23:28:59 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Sun, 08 Dec 2024 23:28:59 +0000 Received: from list by lists.xenproject.org with outflank-mailman.850927.1265144 (Exim 4.92) (envelope-from ) id 1tKQh6-00005R-RF; Sun, 08 Dec 2024 23:28:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 850927.1265144; Sun, 08 Dec 2024 23:28:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tKQh6-00005K-Mv; Sun, 08 Dec 2024 23:28:08 +0000 Received: by outflank-mailman (input) for mailman id 850927; Sun, 08 Dec 2024 23:28:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tKQh5-00005E-HH for xen-users@lists.xenproject.org; Sun, 08 Dec 2024 23:28:07 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 121b0736-b5bc-11ef-a0d5-8be0dac302b0; Mon, 09 Dec 2024 00:28:04 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:8ac4:0:0:0:0:f7]) by mailhost.m5p.com (8.18.1/8.17.1) with ESMTPS id 4B8NRjfB003479 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 8 Dec 2024 18:27:50 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.18.1/8.15.2/Submit) id 4B8NRiHT003478; Sun, 8 Dec 2024 15:27:44 -0800 (PST) (envelope-from ehem) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 121b0736-b5bc-11ef-a0d5-8be0dac302b0 Date: Sun, 8 Dec 2024 15:27:44 -0800 From: Elliott Mitchell To: Mike Cc: Paul Leiber , xen-users@lists.xenproject.org Subject: Re: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on mattapan.m5p.com On Sun, Dec 08, 2024 at 02:03:58AM +0000, Mike wrote: > Elliott Mitchell wrote: > > > Did you in fact look at the output of `xl info` > > Yes. > > > Are you instead guessing based on what you think your > > computer's setup is? > > "313" is a guess based on what I recall changing, since I don't have that > configuration anymore. > > > > dom0_mem=4G;max:4G dom0_max_vcpus=4 dom0_vcpus_pin xpti=dom0=false,domu=true no-real-mode edd=off > > > > Okay, but 4GB for domain 0 strikes me as an odd value. If this computer > > is primarily a server then 1GB should be plenty. > > OK. But that's scaled back from the default, all 64GB of the memory and > ballooning. I can afford it for now. I can also easily add 64GB to the > machine. Okay. Mostly I was asking questions aimed at ruling out memory exhaustion as a possibility. Your numbers are a bit inconsistent, but with those sorts of numbers that seems unlikely. > > Is > > this computer primarily your desktop and you're using Xen to generate VMs > > to experiment with Kubernetes? > > It's a personal server, no GUI. I'm migrating my services to Kubernetes. Okay, 4GB during setup. For a pure VM server exhausting 4GB is quite difficult for the control domain. > > the moment you have a VM machine, it will start eating your network. > > I don't expect much in that regard. I'll probably have a few dozen > Kubernetes pods (VMs), and Calico (a Kubernetes network plugin) handles > all the inter-node (inter-domU) traffic within the domU. But there will > be communicaton between every pod and the control node (the other domU), > and that would pass through the xenbr0. > > > FYI, I've been using Xen for, idk, 15-20 years? It's mostly been set-it- > and-forget-it. Okay, the easier to diagnose issues seem unlikely at this point. Unfortunately this means figuring this out is into hard stages. In order to eliminate software (Xen) bugs, what is really needed is to break out a null modem cable. Then hook up another computer or something else capable of functioning as a dumb terminal (ha! old tech to the rescue). Serial ports are rather more effective for observing Xen panicing. Any possibility the computer's power supply is near its limit? What could explain the sudden reset is processor usage goes up and pushes something power related beyond its limit. For the heck of it, you might try adding "cpuidle" to Xen's command-line. Then ensure the kernel module "xen_acpi_processor" is loaded. If this helps then it could be power or perhaps processor heatsink. -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445 From xen-users-bounces@lists.xenproject.org Mon Dec 09 00:45:06 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Mon, 09 Dec 2024 00:45:06 +0000 Received: from list by lists.xenproject.org with outflank-mailman.850964.1265159 (Exim 4.92) (envelope-from ) id 1tKRsv-0001rJ-S7; Mon, 09 Dec 2024 00:44:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 850964.1265159; Mon, 09 Dec 2024 00:44:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tKRsv-0001rC-PP; Mon, 09 Dec 2024 00:44:25 +0000 Received: by outflank-mailman (input) for mailman id 850964; Mon, 09 Dec 2024 00:44:23 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tKRst-0001r4-N7 for xen-users@lists.xenproject.org; Mon, 09 Dec 2024 00:44:23 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ba205672-b5c6-11ef-a0d5-8be0dac302b0; Mon, 09 Dec 2024 01:44:21 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: ba205672-b5c6-11ef-a0d5-8be0dac302b0 Date: Mon, 9 Dec 2024 00:41:27 +0000 From: Mike To: Elliott Mitchell Cc: Paul Leiber , xen-users@lists.xenproject.org Subject: Re: domU memory exceeded =?us-ascii?B?PT89?= =?us-ascii?Q?=3E?= spontaneous reboots Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: > In order to eliminate software (Xen) bugs, what is really needed is to > break out a null modem cable. Then hook up another computer or something > else capable of functioning as a dumb terminal (ha! old tech to the > rescue). Serial ports are rather more effective for observing Xen > panicing. The motherboard doesn't have a serial port on its backpanel, just jumpers for attaching one on the board. And the case doesn't offer a place to mount a serial port. I'm not inclined to mess with the system to that degree at this point. It's functioning for now. > Any possibility the computer's power supply is near its limit? What > could explain the sudden reset is processor usage goes up and pushes > something power related beyond its limit. I sent you a link to one blog post. There's another post that covers stress testing, which I think eliminates the idea of a power draw surge causing the reboot. There you can see that the system was running with all cores loaded. The CPU is the main power draw in this system. Yes, I added an NVMe drive and a couple SSDs, but that's nothing compared to the CPU. And currently, the CPU load is quite small still. In any case, I think I spec'd the system with an adequate PSU. It also shows that the cores don't exceed 60C under load, which I think I mentioned already. > For the heck of it, you might try adding "cpuidle" to Xen's command-line. > Then ensure the kernel module "xen_acpi_processor" is loaded. If this > helps then it could be power or perhaps processor heatsink. I'll put that on my to-do list, if just for the power savings. Thanks for thinking about it. I was just recording the issue, not expecting a resolution given the low level of the behavior. From xen-users-bounces@lists.xenproject.org Tue Dec 10 19:49:35 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 10 Dec 2024 19:49:35 +0000 Received: from list by lists.xenproject.org with outflank-mailman.852986.1266612 (Exim 4.92) (envelope-from ) id 1tL6Dt-00014D-Qj; Tue, 10 Dec 2024 19:48:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 852986.1266612; Tue, 10 Dec 2024 19:48:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tL6Dt-000146-Nr; Tue, 10 Dec 2024 19:48:45 +0000 Received: by outflank-mailman (input) for mailman id 852986; Tue, 10 Dec 2024 19:48:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tL6Ds-000140-Vm for xen-users@lists.xenproject.org; Tue, 10 Dec 2024 19:48:44 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c1004e5c-b72f-11ef-99a3-01e77a169b0f; Tue, 10 Dec 2024 20:48:42 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: c1004e5c-b72f-11ef-99a3-01e77a169b0f Date: Tue, 10 Dec 2024 19:45:50 +0000 From: Mike To: Elliott Mitchell Cc: xen-users@lists.xenproject.org Subject: cpuidle flag Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Elliott Mitchell wrote: > try adding "cpuidle" to Xen's command-line. This does nothing. Per the wiki, it's enabled by default since Xen 3.4. I confirmed that with `xenpm get-cpuidle-states`. From xen-users-bounces@lists.xenproject.org Tue Dec 10 21:15:39 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 10 Dec 2024 21:15:39 +0000 Received: from list by lists.xenproject.org with outflank-mailman.853037.1266628 (Exim 4.92) (envelope-from ) id 1tL7ZM-0003Aq-31; Tue, 10 Dec 2024 21:15:00 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 853037.1266628; Tue, 10 Dec 2024 21:15:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tL7ZM-0003Aj-0B; Tue, 10 Dec 2024 21:15:00 +0000 Received: by outflank-mailman (input) for mailman id 853037; Tue, 10 Dec 2024 21:14:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tL7ZK-0003Ad-Rd for xen-users@lists.xenproject.org; Tue, 10 Dec 2024 21:14:58 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id cd35f128-b73b-11ef-99a3-01e77a169b0f; Tue, 10 Dec 2024 22:14:55 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:8ac4:0:0:0:0:f7]) by mailhost.m5p.com (8.18.1/8.17.1) with ESMTPS id 4BALDaVN014079 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 10 Dec 2024 16:13:42 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.18.1/8.15.2/Submit) id 4BALDai4014078; Tue, 10 Dec 2024 13:13:36 -0800 (PST) (envelope-from ehem) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: cd35f128-b73b-11ef-99a3-01e77a169b0f Date: Tue, 10 Dec 2024 13:13:36 -0800 From: Elliott Mitchell To: Mike Cc: xen-users@lists.xenproject.org Subject: Re: cpuidle flag Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on mattapan.m5p.com On Tue, Dec 10, 2024 at 07:45:50PM +0000, Mike wrote: > Elliott Mitchell wrote: > > try adding "cpuidle" to Xen's command-line. > > This does nothing. Per the wiki, it's enabled by default since Xen 3.4. > I confirmed that with `xenpm get-cpuidle-states`. Are you sure? With 4.11 I had to deliberately enable "cpuidle" AND load the "xen_acpi_processor.ko" module. Otherwise Xen only had C1 (hlt instruction) and not C2 or greater. Other trick is "xen_acpi_processor.ko" only enables C-states for which Domain 0 had corresponding vCPUs. Presently I'm having to give Domain 0 the maximum vCPUs, then use `xl vcpu-set` to offline most of them. I haven't yet chased this down. Debug information from "xen_acpi_processor.ko" suggests there are 2 loops for uploading ACPI states. The first one uploads both P and C states, but only runs on processors for which Domain 0 has a corresponding vCPU. The second runs for processors which have no Domain 0 vCPU, but only uploads P-states. -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445 From xen-users-bounces@lists.xenproject.org Tue Dec 10 21:46:11 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 10 Dec 2024 21:46:11 +0000 Received: from list by lists.xenproject.org with outflank-mailman.853078.1266644 (Exim 4.92) (envelope-from ) id 1tL83A-0007J5-IM; Tue, 10 Dec 2024 21:45:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 853078.1266644; Tue, 10 Dec 2024 21:45:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tL83A-0007Iy-Fa; Tue, 10 Dec 2024 21:45:48 +0000 Received: by outflank-mailman (input) for mailman id 853078; Tue, 10 Dec 2024 21:45:47 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tL839-0007Is-N4 for xen-users@lists.xenproject.org; Tue, 10 Dec 2024 21:45:47 +0000 Received: from out.good-with-numbers.com (out.good-with-numbers.com [70.36.235.100]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 15f6e395-b740-11ef-a0d5-8be0dac302b0; Tue, 10 Dec 2024 22:45:45 +0100 (CET) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 15f6e395-b740-11ef-a0d5-8be0dac302b0 Date: Tue, 10 Dec 2024 21:42:12 +0000 From: Mike To: Elliott Mitchell Cc: xen-users@lists.xenproject.org Subject: Re: cpuidle flag Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Elliott Mitchell wrote: > Are you sure? > Other trick is "xen_acpi_processor.ko" only enables C-states for which > Domain 0 had corresponding vCPUs. Without the flag, `xenpm get-cpuidle-states` returns --- total C-states : 4 --- for the dom0 CPUs (which are pinned). But the others only seem to have C0, C1. `lsmod | grep xen_acpi_processor` returns a line. Didn't ever enable it, as far as I recall. From xen-users-bounces@lists.xenproject.org Sat Dec 14 02:58:19 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Sat, 14 Dec 2024 02:58:19 +0000 Received: from list by lists.xenproject.org with outflank-mailman.857273.1269658 (Exim 4.92) (envelope-from ) id 1tMILE-0000rw-Dj; Sat, 14 Dec 2024 02:57:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 857273.1269658; Sat, 14 Dec 2024 02:57:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tMILE-0000rp-7u; Sat, 14 Dec 2024 02:57:16 +0000 Received: by outflank-mailman (input) for mailman id 857273; Sat, 14 Dec 2024 02:57:14 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tMILC-0000rj-H9 for xen-users@lists.xenproject.org; Sat, 14 Dec 2024 02:57:14 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1bd050ad-b9c7-11ef-99a3-01e77a169b0f; Sat, 14 Dec 2024 03:57:11 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7]) by mailhost.m5p.com (8.18.1/8.17.1) with ESMTPS id 4BE2uuEC032241 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 13 Dec 2024 21:57:02 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.18.1/8.15.2/Submit) id 4BE2utje032240; Fri, 13 Dec 2024 18:56:55 -0800 (PST) (envelope-from ehem) X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 1bd050ad-b9c7-11ef-99a3-01e77a169b0f Date: Fri, 13 Dec 2024 18:56:55 -0800 From: Elliott Mitchell To: Mike Cc: xen-users@lists.xenproject.org Subject: Re: cpuidle flag Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=0.4 required=10.0 tests=KHOP_HELO_FCRDNS autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on mattapan.m5p.com On Tue, Dec 10, 2024 at 09:42:12PM +0000, Mike wrote: > Elliott Mitchell wrote: > > Are you sure? > > > Other trick is "xen_acpi_processor.ko" only enables C-states for which > > Domain 0 had corresponding vCPUs. > > Without the flag, `xenpm get-cpuidle-states` returns > > --- > total C-states : 4 > --- > > for the dom0 CPUs (which are pinned). But the others only seem to have C0, > C1. > > `lsmod | grep xen_acpi_processor` returns a line. Didn't ever enable it, as > far as I recall. If you're on Debian, it got added to the scripts due to my bringing it to the maintainer's attention. What you've mentioned matches what I've encountered. I'm fairly sure this is a bug in xen_acpi_processor.ko. I've almost got enough information to point to what is happening. Just I need to wait for my next system restart (kernel update) before pointing the finger. My present workaround is to boot Domain 0 with full processor, but in /etc/rc.local do `xl vcpu-set 0 ###`. Linux gracefully handles hot processor removal, so this is adaquate for the moment. -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445 From xen-users-bounces@lists.xenproject.org Tue Dec 17 12:19:24 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 17 Dec 2024 12:19:24 +0000 Received: from list by lists.xenproject.org with outflank-mailman.858915.1271152 (Exim 4.92) (envelope-from ) id 1tNWXM-0002Ym-C7; Tue, 17 Dec 2024 12:18:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 858915.1271152; Tue, 17 Dec 2024 12:18:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tNWXM-0002Yf-93; Tue, 17 Dec 2024 12:18:52 +0000 Received: by outflank-mailman (input) for mailman id 858915; Tue, 17 Dec 2024 12:18:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tNWXL-0002GC-Ea for xen-users@lists.xen.org; Tue, 17 Dec 2024 12:18:51 +0000 Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 10800b33-bc71-11ef-a0d6-8be0dac302b0; Tue, 17 Dec 2024 13:18:48 +0100 (CET) Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.96) (envelope-from ) id 1tNWXC-0081kR-16; Tue, 17 Dec 2024 12:18:42 +0000 Received: from andrewcoop by xenbits.xenproject.org with local (Exim 4.96) (envelope-from ) id 1tNWXC-00E24k-1b; Tue, 17 Dec 2024 12:18:42 +0000 X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 10800b33-bc71-11ef-a0d6-8be0dac302b0 Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8" Content-Transfer-Encoding: binary MIME-Version: 1.0 X-Mailer: MIME-tools 5.510 (Entity 5.510) To: xen-announce@lists.xen.org, xen-devel@lists.xen.org, xen-users@lists.xen.org, oss-security@lists.openwall.com From: Xen.org security team CC: Xen.org security team Subject: Xen Security Advisory 465 v3 (CVE-2024-53240) - Backend can crash Linux netfront Message-Id: Date: Tue, 17 Dec 2024 12:18:42 +0000 --=separator Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Xen Security Advisory CVE-2024-53240 / XSA-465 version 3 Backend can crash Linux netfront UPDATES IN VERSION 3 ==================== Public release. ISSUE DESCRIPTION ================= After a suspend/resume cycle of a Linux guest (e.g. via "virsh dompmsuspend"/ "virsh dompmwakeup") a malicious network backend can crash the guest via a NULL-pointer dereference in the guest's xen-netfront driver. During the resume operation the xen-netfront driver will release some data structures used for communication with the backend, in order to reallocate these data structures with possibly different parameters specified by the backend. If the backend is triggering a network device removal in the guest before any network I/O has happened, the NULL-pointer dereference may happen, causing a crash of the guest. IMPACT ====== In setups with non-trusted network backends (e.g. when using untrusted network driver domains) suspend/resume cycles of guests can result in those guests being crashed by a malicious network backend. VULNERABLE SYSTEMS ================== Only systems with non-trusted network backends are vulnerable. As far as known only Linux guests with the fix for CVE-2022-48969 applied are vulnerable (this includes all kernel versions from 6.1 onwards). All guest types (x86 PV, x86 PVH/HVM and Arm32/Arm64) are vulnerable. MITIGATION ========== Not doing guest suspend/resume cycles will avoid the vulnerability. Using emulated NICs instead of PV ones will avoid the vulnerability. CREDITS ======= This issue was discovered by Marek Marczykowski-Górecki of Invisible Things Lab. RESOLUTION ========== Applying the attached patch resolves this issue. xsa465-linux.patch Linux $ sha256sum xsa465* 7207a22e1e70d0b00278d90e797313bee9d72a968ddd38464b90f0612667826e xsa465-linux.patch $ DEPLOYMENT DURING EMBARGO ========================= Deployment of patches or mitigations is NOT permitted (except where all the affected systems and VMs are administered and used only by organisations which are members of the Xen Project Security Issues Predisclosure List). Specifically, deployment on public cloud systems is NOT permitted. This is because the patches need to be applied to the guests and using emulated NICs is a guest visible configuration change. Deployment is permitted only AFTER the embargo ends. (Note: this during-embargo deployment notice is retained in post-embargo publicly released Xen Project advisories, even though it is then no longer applicable. This is to enable the community to have oversight of the Xen Project Security Team's decisionmaking.) For more information about permissible uses of embargoed information, consult the Xen Project community's agreed Security Policy: http://www.xenproject.org/security-policy.html -----BEGIN PGP SIGNATURE----- iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmdhaw0MHHBncEB4ZW4u b3JnAAoJEIP+FMlX6CvZ1kYH/3u0RtjvoOLf2CTFAPhBfgVXZ0nbaQAVeVY14OXL 3WAOQzrspobwSJtVUqRCg14NllEkM2ityeAlussY++b9BFW7nqxji9yL/rSMpuPh vsH/sDByBSUYxpaw/LgbkZVvhRq3vbK6E7fnXCw8BO9LYA+uTZRf4P6PRe0JeQtz t0IyHsECXaPoSWzX18OtSrg1JFYhgBqB9vK4rKMvMjPpqZDIKlEgIpFwNlywZ6jx H6T3CCKUPUZqmVegxJtXIof3STEr9bzd4StPaUrRXfToOg5ZsknUkari0Nr8xW27 mcTZaFVWgWwfI0irMs9jTp2agfQ6T+yptA8ZfM3J7kGvGcc= =+A6o -----END PGP SIGNATURE----- --=separator Content-Type: application/octet-stream; name="xsa465-linux.patch" Content-Disposition: attachment; filename="xsa465-linux.patch" Content-Transfer-Encoding: base64 RnJvbSBmOTI0NGZiNTVmMzczNTZmNzVjNzM5YzU3MzIzZDk0MjJkN2FhMGY4 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IFRodSwgNyBOb3YgMjAyNCAxNjox NzowMCArMDEwMApTdWJqZWN0OiB4ZW4vbmV0ZnJvbnQ6IGZpeCBjcmFzaCB3 aGVuIHJlbW92aW5nIGRldmljZQpNSU1FLVZlcnNpb246IDEuMApDb250ZW50 LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9VVRGLTgKQ29udGVudC1UcmFu c2Zlci1FbmNvZGluZzogOGJpdAoKV2hlbiByZW1vdmluZyBhIG5ldGZyb250 IGRldmljZSBkaXJlY3RseSBhZnRlciBhIHN1c3BlbmQvcmVzdW1lIGN5Y2xl Cml0IG1pZ2h0IGhhcHBlbiB0aGF0IHRoZSBxdWV1ZXMgaGF2ZSBub3QgYmVl biBzZXR1cCBhZ2FpbiwgY2F1c2luZyBhCmNyYXNoIGR1cmluZyB0aGUgYXR0 ZW1wdCB0byBzdG9wIHRoZSBxdWV1ZXMgYW5vdGhlciB0aW1lLgoKRml4IHRo YXQgYnkgY2hlY2tpbmcgdGhlIHF1ZXVlcyBhcmUgZXhpc3RpbmcgYmVmb3Jl IHRyeWluZyB0byBzdG9wCnRoZW0uCgpUaGlzIGlzIFhTQS00NjUgLyBDVkUt MjAyNC01MzI0MC4KClJlcG9ydGVkLWJ5OiBNYXJlayBNYXJjenlrb3dza2kt R8OzcmVja2kgPG1hcm1hcmVrQGludmlzaWJsZXRoaW5nc2xhYi5jb20+CkZp eGVzOiBkNTBiNzkxNGZhZTAgKCJ4ZW4tbmV0ZnJvbnQ6IEZpeCBOVUxMIHNy aW5nIGFmdGVyIGxpdmUgbWlncmF0aW9uIikKU2lnbmVkLW9mZi1ieTogSnVl cmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPgotLS0KIGRyaXZlcnMvbmV0 L3hlbi1uZXRmcm9udC5jIHwgNSArKysrLQogMSBmaWxlIGNoYW5nZWQsIDQg aW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL2Ry aXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jIGIvZHJpdmVycy9uZXQveGVuLW5l dGZyb250LmMKaW5kZXggNDI2NWMxY2QwZmY3Li42M2ZlNTFkMGU2NGQgMTAw NjQ0Ci0tLSBhL2RyaXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCisrKyBiL2Ry aXZlcnMvbmV0L3hlbi1uZXRmcm9udC5jCkBAIC04NjcsNyArODY3LDcgQEAg c3RhdGljIG5ldGRldl90eF90IHhlbm5ldF9zdGFydF94bWl0KHN0cnVjdCBz a19idWZmICpza2IsIHN0cnVjdCBuZXRfZGV2aWNlICpkZXYKIHN0YXRpYyBp bnQgeGVubmV0X2Nsb3NlKHN0cnVjdCBuZXRfZGV2aWNlICpkZXYpCiB7CiAJ c3RydWN0IG5ldGZyb250X2luZm8gKm5wID0gbmV0ZGV2X3ByaXYoZGV2KTsK LQl1bnNpZ25lZCBpbnQgbnVtX3F1ZXVlcyA9IGRldi0+cmVhbF9udW1fdHhf cXVldWVzOworCXVuc2lnbmVkIGludCBudW1fcXVldWVzID0gbnAtPnF1ZXVl cyA/IGRldi0+cmVhbF9udW1fdHhfcXVldWVzIDogMDsKIAl1bnNpZ25lZCBp bnQgaTsKIAlzdHJ1Y3QgbmV0ZnJvbnRfcXVldWUgKnF1ZXVlOwogCW5ldGlm X3R4X3N0b3BfYWxsX3F1ZXVlcyhucC0+bmV0ZGV2KTsKQEAgLTg4Miw2ICs4 ODIsOSBAQCBzdGF0aWMgdm9pZCB4ZW5uZXRfZGVzdHJveV9xdWV1ZXMoc3Ry dWN0IG5ldGZyb250X2luZm8gKmluZm8pCiB7CiAJdW5zaWduZWQgaW50IGk7 CiAKKwlpZiAoIWluZm8tPnF1ZXVlcykKKwkJcmV0dXJuOworCiAJZm9yIChp ID0gMDsgaSA8IGluZm8tPm5ldGRldi0+cmVhbF9udW1fdHhfcXVldWVzOyBp KyspIHsKIAkJc3RydWN0IG5ldGZyb250X3F1ZXVlICpxdWV1ZSA9ICZpbmZv LT5xdWV1ZXNbaV07CiAKLS0gCjIuNDMuMAoK --=separator-- From xen-users-bounces@lists.xenproject.org Tue Dec 17 12:19:24 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 17 Dec 2024 12:19:24 +0000 Received: from list by lists.xenproject.org with outflank-mailman.858919.1271195 (Exim 4.92) (envelope-from ) id 1tNWXR-0003VW-1G; Tue, 17 Dec 2024 12:18:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 858919.1271195; Tue, 17 Dec 2024 12:18:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tNWXQ-0003U4-Rh; Tue, 17 Dec 2024 12:18:56 +0000 Received: by outflank-mailman (input) for mailman id 858919; Tue, 17 Dec 2024 12:18:55 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tNWXP-0002GC-FA for xen-users@lists.xen.org; Tue, 17 Dec 2024 12:18:55 +0000 Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 1324b242-bc71-11ef-a0d6-8be0dac302b0; Tue, 17 Dec 2024 13:18:51 +0100 (CET) Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.96) (envelope-from ) id 1tNWXG-0081kk-23; Tue, 17 Dec 2024 12:18:46 +0000 Received: from andrewcoop by xenbits.xenproject.org with local (Exim 4.96) (envelope-from ) id 1tNWXG-00E268-2p; Tue, 17 Dec 2024 12:18:46 +0000 X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 1324b242-bc71-11ef-a0d6-8be0dac302b0 Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8" Content-Transfer-Encoding: binary MIME-Version: 1.0 X-Mailer: MIME-tools 5.510 (Entity 5.510) To: xen-announce@lists.xen.org, xen-devel@lists.xen.org, xen-users@lists.xen.org, oss-security@lists.openwall.com From: Xen.org security team CC: Xen.org security team Subject: Xen Security Advisory 466 v3 (CVE-2024-53241) - Xen hypercall page unsafe against speculative attacks Message-Id: Date: Tue, 17 Dec 2024 12:18:46 +0000 --=separator Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Xen Security Advisory CVE-2024-53241 / XSA-466 version 3 Xen hypercall page unsafe against speculative attacks UPDATES IN VERSION 3 ==================== Update of patch 5, public release. ISSUE DESCRIPTION ================= Xen guests need to use different processor instructions to make explicit calls into the Xen hypervisor depending on guest type and/or CPU vendor. In order to hide those differences, the hypervisor can fill a hypercall page with the needed instruction sequences, allowing the guest operating system to call into the hypercall page instead of having to choose the correct instructions. The hypercall page contains whole functions, which are written by the hypervisor and executed by the guest. With the lack of an interface between the guest OS and the hypervisor specifying how a potential modification of those functions should look like, the Xen hypervisor has no knowledge how any potential mitigation should look like or which hardening features should be put into place. This results in potential vulnerabilities if the guest OS is using any speculative mitigation that performs a compiler transform on "ret" instructions in order to work (e.g. the Linux kernel rethunk or safe-ret mitigations). Furthermore, the hypercall page has no provision for Control-flow Integrity schemes (e.g. kCFI/CET-IBT/FineIBT), and will simply malfunction in such configurations. IMPACT ====== Some mitigations for hardware vulnerabilities the guest OS is relying on to work might not be fully functional, resulting in e.g. guest user processes being able to read data they ought not have access to. VULNERABLE SYSTEMS ================== Only x86 systems are potentially vulnerable, Arm systems are not vulnerable. All guest types (PV, PVH and HVM) are potentially vulnerable. Linux guests are known to be vulnerable, guests using other operating systems might be vulnerable, too. MITIGATION ========== Running only Linux guest kernels not relying on "ret" assembler instruction patching (kernel config option CONFIG_MITIGATION_RETHUNK/CONFIG_RETHUNK disabled) will avoid the vulnerability, as long as this option isn't required to be safe on the underlying hardware. CREDITS ======= This issue was discovered by Andrew Cooper of XenServer. RESOLUTION ========== Applying the set of attached patches resolves this issue. The patch to Xen is simply a documentation update to clarify that an OS author might not want to use a hypercall page. xsa466-linux-*.patch Linux xsa466-xen.patch xen-unstable $ sha256sum xsa466* 498fb2538f650d694bbd6b7d2333dcf9a12d0bdfcba65257a7d14c88f5b86801 xsa466-linux-01.patch 1e0d5f68d1cb4a0ef8914ae6bdeb4e18bae94c6d19659708ad707da784c0aa5c xsa466-linux-02.patch b3056b34c1565f901cb4ba11c03a51d4f045b5de7cd16c6e510e0bcee8cc6cd7 xsa466-linux-03.patch 0215e56739ab5b0d0ec0125f3d1806c3a0a0dcb3f562014f59b5145184a41467 xsa466-linux-04.patch 314e67060ab4f47883cf2b124d54ce3cd4b0363f0545ad907a7b754a4405aacd xsa466-linux-05.patch adbef75416379d96ebb72463872f993e9d8b7d119091480ad1e70fd448481733 xsa466-linux-06.patch 36874014cee5d5213610a6ffdd0e3e67d0258d28f2587b8470fdd0cef96e5013 xsa466-linux-07.patch 367f981ef8adc11b99cc6999b784305bcdcd55db0358fd6a2171509bf7f64345 xsa466-xen.patch $ DEPLOYMENT DURING EMBARGO ========================= Deployment of patches or mitigations is NOT permitted (except where all the affected systems and VMs are administered and used only by organisations which are members of the Xen Project Security Issues Predisclosure List). Specifically, deployment on public cloud systems is NOT permitted. This is because the mitigation or patches need to be applied to the guests. Deployment is permitted only AFTER the embargo ends. (Note: this during-embargo deployment notice is retained in post-embargo publicly released Xen Project advisories, even though it is then no longer applicable. This is to enable the community to have oversight of the Xen Project Security Team's decisionmaking.) For more information about permissible uses of embargoed information, consult the Xen Project community's agreed Security Policy: http://www.xenproject.org/security-policy.html -----BEGIN PGP SIGNATURE----- iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmdhaxAMHHBncEB4ZW4u b3JnAAoJEIP+FMlX6CvZUeMH/0Qkn9G8iQWJ0fHMCxvd1lcr3RNWK5GfXqlAZvuJ YRQMDslYCzuvzrkLnoe/P/zSSs+omEYMcOVsJCBkTqePs8yIdqwBvBfZ79I1htIu IdyJt8SE5+b70ZlumQJ8ef1Za3lp8bxvEZVa8GIokOu0Ef1iqUKNl7tQgIoQjOUH bV/1sFN5MNFsUshOW5DnLiRrE8j0/0nfbzHPu5H9S2B4eN38oPmTabvZG/IHky8R VFyTvqFrKZONDDhdxyFE9PBOtP6Bu3EV+Emmxb3Q84l2oEIqgab0Xxj4QGBTuLMn PPcU5/D6Giqx3jBMdrkMAXtBuXBYO/inqsX1IJLic9W13+A= =wlOW -----END PGP SIGNATURE----- --=separator Content-Type: application/octet-stream; name="xsa466-linux-01.patch" Content-Disposition: attachment; filename="xsa466-linux-01.patch" Content-Transfer-Encoding: base64 RnJvbSBlZmJjZDYxZDliZWJiNzcxYzgzNmEzYjhiZmNlZDgxNjU2MzNkYjdj IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IFRodSwgMTcgT2N0IDIwMjQgMDg6 Mjk6NDggKzAyMDAKU3ViamVjdDogeDg2OiBtYWtlIGdldF9jcHVfdmVuZG9y KCkgYWNjZXNzaWJsZSBmcm9tIFhlbiBjb2RlCgpJbiBvcmRlciB0byBiZSBh YmxlIHRvIGRpZmZlcmVudGlhdGUgYmV0d2VlbiBBTUQgYW5kIEludGVsIGJh c2VkCnN5c3RlbXMgZm9yIHZlcnkgZWFybHkgaHlwZXJjYWxscyB3aXRob3V0 IGhhdmluZyB0byByZWx5IG9uIHRoZSBYZW4KaHlwZXJjYWxsIHBhZ2UsIG1h a2UgZ2V0X2NwdV92ZW5kb3IoKSBub24tc3RhdGljLgoKUmVmYWN0b3IgZWFy bHlfY3B1X2luaXQoKSBmb3IgdGhlIHNhbWUgcmVhc29uIGJ5IHNwbGl0dGlu ZyBvdXQgdGhlCmxvb3AgaW5pdGlhbGl6aW5nIGNwdV9kZXZzKCkgaW50byBh biBleHRlcm5hbGx5IGNhbGxhYmxlIGZ1bmN0aW9uLgoKVGhpcyBpcyBwYXJ0 IG9mIFhTQS00NjYgLyBDVkUtMjAyNC01MzI0MS4KClJlcG9ydGVkLWJ5OiBB bmRyZXcgQ29vcGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPgpTaWdu ZWQtb2ZmLWJ5OiBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+Ci0t LQogYXJjaC94ODYvaW5jbHVkZS9hc20vcHJvY2Vzc29yLmggfCAgMiArKwog YXJjaC94ODYva2VybmVsL2NwdS9jb21tb24uYyAgICAgfCAzOCArKysrKysr KysrKysrKysrKystLS0tLS0tLS0tLS0tLQogMiBmaWxlcyBjaGFuZ2VkLCAy NCBpbnNlcnRpb25zKCspLCAxNiBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQg YS9hcmNoL3g4Ni9pbmNsdWRlL2FzbS9wcm9jZXNzb3IuaCBiL2FyY2gveDg2 L2luY2x1ZGUvYXNtL3Byb2Nlc3Nvci5oCmluZGV4IGMwOTc1ODE1OTgwYy4u MjBlNjAwOTM4MWVkIDEwMDY0NAotLS0gYS9hcmNoL3g4Ni9pbmNsdWRlL2Fz bS9wcm9jZXNzb3IuaAorKysgYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS9wcm9j ZXNzb3IuaApAQCAtMjMwLDYgKzIzMCw4IEBAIHN0YXRpYyBpbmxpbmUgdW5z aWduZWQgbG9uZyBsb25nIGwxdGZfcGZuX2xpbWl0KHZvaWQpCiAJcmV0dXJu IEJJVF9VTEwoYm9vdF9jcHVfZGF0YS54ODZfY2FjaGVfYml0cyAtIDEgLSBQ QUdFX1NISUZUKTsKIH0KIAordm9pZCBpbml0X2NwdV9kZXZzKHZvaWQpOwor dm9pZCBnZXRfY3B1X3ZlbmRvcihzdHJ1Y3QgY3B1aW5mb194ODYgKmMpOwog ZXh0ZXJuIHZvaWQgZWFybHlfY3B1X2luaXQodm9pZCk7CiBleHRlcm4gdm9p ZCBpZGVudGlmeV9zZWNvbmRhcnlfY3B1KHN0cnVjdCBjcHVpbmZvX3g4NiAq KTsKIGV4dGVybiB2b2lkIHByaW50X2NwdV9pbmZvKHN0cnVjdCBjcHVpbmZv X3g4NiAqKTsKZGlmZiAtLWdpdCBhL2FyY2gveDg2L2tlcm5lbC9jcHUvY29t bW9uLmMgYi9hcmNoL3g4Ni9rZXJuZWwvY3B1L2NvbW1vbi5jCmluZGV4IGE1 YzI4OTc1YzYwOC4uM2U5MDM3NjkwODE0IDEwMDY0NAotLS0gYS9hcmNoL3g4 Ni9rZXJuZWwvY3B1L2NvbW1vbi5jCisrKyBiL2FyY2gveDg2L2tlcm5lbC9j cHUvY29tbW9uLmMKQEAgLTg2Nyw3ICs4NjcsNyBAQCBzdGF0aWMgdm9pZCBj cHVfZGV0ZWN0X3RsYihzdHJ1Y3QgY3B1aW5mb194ODYgKmMpCiAJCXRsYl9s bGRfNG1bRU5UUklFU10sIHRsYl9sbGRfMWdbRU5UUklFU10pOwogfQogCi1z dGF0aWMgdm9pZCBnZXRfY3B1X3ZlbmRvcihzdHJ1Y3QgY3B1aW5mb194ODYg KmMpCit2b2lkIGdldF9jcHVfdmVuZG9yKHN0cnVjdCBjcHVpbmZvX3g4NiAq YykKIHsKIAljaGFyICp2ID0gYy0+eDg2X3ZlbmRvcl9pZDsKIAlpbnQgaTsK QEAgLTE2NDksMTUgKzE2NDksMTEgQEAgc3RhdGljIHZvaWQgX19pbml0IGVh cmx5X2lkZW50aWZ5X2NwdShzdHJ1Y3QgY3B1aW5mb194ODYgKmMpCiAJZGV0 ZWN0X25vcGwoKTsKIH0KIAotdm9pZCBfX2luaXQgZWFybHlfY3B1X2luaXQo dm9pZCkKK3ZvaWQgX19pbml0IGluaXRfY3B1X2RldnModm9pZCkKIHsKIAlj b25zdCBzdHJ1Y3QgY3B1X2RldiAqY29uc3QgKmNkZXY7CiAJaW50IGNvdW50 ID0gMDsKIAotI2lmZGVmIENPTkZJR19QUk9DRVNTT1JfU0VMRUNUCi0JcHJf aW5mbygiS0VSTkVMIHN1cHBvcnRlZCBjcHVzOlxuIik7Ci0jZW5kaWYKLQog CWZvciAoY2RldiA9IF9feDg2X2NwdV9kZXZfc3RhcnQ7IGNkZXYgPCBfX3g4 Nl9jcHVfZGV2X2VuZDsgY2RldisrKSB7CiAJCWNvbnN0IHN0cnVjdCBjcHVf ZGV2ICpjcHVkZXYgPSAqY2RldjsKIApAQCAtMTY2NSwyMCArMTY2MSwzMCBA QCB2b2lkIF9faW5pdCBlYXJseV9jcHVfaW5pdCh2b2lkKQogCQkJYnJlYWs7 CiAJCWNwdV9kZXZzW2NvdW50XSA9IGNwdWRldjsKIAkJY291bnQrKzsKKwl9 Cit9CiAKK3ZvaWQgX19pbml0IGVhcmx5X2NwdV9pbml0KHZvaWQpCit7CiAj aWZkZWYgQ09ORklHX1BST0NFU1NPUl9TRUxFQ1QKLQkJewotCQkJdW5zaWdu ZWQgaW50IGo7Ci0KLQkJCWZvciAoaiA9IDA7IGogPCAyOyBqKyspIHsKLQkJ CQlpZiAoIWNwdWRldi0+Y19pZGVudFtqXSkKLQkJCQkJY29udGludWU7Ci0J CQkJcHJfaW5mbygiICAlcyAlc1xuIiwgY3B1ZGV2LT5jX3ZlbmRvciwKLQkJ CQkJY3B1ZGV2LT5jX2lkZW50W2pdKTsKLQkJCX0KLQkJfQorCXVuc2lnbmVk IGludCBpLCBqOworCisJcHJfaW5mbygiS0VSTkVMIHN1cHBvcnRlZCBjcHVz OlxuIik7CiAjZW5kaWYKKworCWluaXRfY3B1X2RldnMoKTsKKworI2lmZGVm IENPTkZJR19QUk9DRVNTT1JfU0VMRUNUCisJZm9yIChpID0gMDsgaSA8IFg4 Nl9WRU5ET1JfTlVNICYmIGNwdV9kZXZzW2ldOyBpKyspIHsKKwkJZm9yIChq ID0gMDsgaiA8IDI7IGorKykgeworCQkJaWYgKCFjcHVfZGV2c1tpXS0+Y19p ZGVudFtqXSkKKwkJCQljb250aW51ZTsKKwkJCXByX2luZm8oIiAgJXMgJXNc biIsIGNwdV9kZXZzW2ldLT5jX3ZlbmRvciwKKwkJCQljcHVfZGV2c1tpXS0+ Y19pZGVudFtqXSk7CisJCX0KIAl9CisjZW5kaWYKKwogCWVhcmx5X2lkZW50 aWZ5X2NwdSgmYm9vdF9jcHVfZGF0YSk7CiB9CiAKLS0gCjIuNDMuMAoK --=separator Content-Type: application/octet-stream; name="xsa466-linux-02.patch" Content-Disposition: attachment; filename="xsa466-linux-02.patch" Content-Transfer-Encoding: base64 RnJvbSBkZGEwMTRiYTU5MzMxZGVlNGYzYjc3M2EwMjBlMTA5OTMyZjRiZDI0 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IEZyaSwgMjkgTm92IDIwMjQgMTU6 NDc6NDkgKzAxMDAKU3ViamVjdDogb2JqdG9vbC94ODY6IGFsbG93IHN5c2Nh bGwgaW5zdHJ1Y3Rpb24KClRoZSBzeXNjYWxsIGluc3RydWN0aW9uIGlzIHVz ZWQgaW4gWGVuIFBWIG1vZGUgZm9yIGRvaW5nIGh5cGVyY2FsbHMuCkFsbG93 IHN5c2NhbGwgdG8gYmUgdXNlZCBpbiB0aGUga2VybmVsIGluIGNhc2UgaXQg aXMgdGFnZ2VkIHdpdGggYW4KdW53aW5kIGhpbnQgZm9yIG9ianRvb2wuCgpU aGlzIGlzIHBhcnQgb2YgWFNBLTQ2NiAvIENWRS0yMDI0LTUzMjQxLgoKUmVw b3J0ZWQtYnk6IEFuZHJldyBDb29wZXIgPGFuZHJldy5jb29wZXIzQGNpdHJp eC5jb20+ClNpZ25lZC1vZmYtYnk6IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0Bz dXNlLmNvbT4KQ28tZGV2ZWxvcGVkLWJ5OiBQZXRlciBaaWpsc3RyYSA8cGV0 ZXJ6QGluZnJhZGVhZC5vcmc+Ci0tLQogdG9vbHMvb2JqdG9vbC9jaGVjay5j IHwgOSArKysrKystLS0KIDEgZmlsZSBjaGFuZ2VkLCA2IGluc2VydGlvbnMo KyksIDMgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvdG9vbHMvb2JqdG9v bC9jaGVjay5jIGIvdG9vbHMvb2JqdG9vbC9jaGVjay5jCmluZGV4IDRjZTE3 NmFkNDExZi4uNzYwNjBkYTc1NWI1IDEwMDY0NAotLS0gYS90b29scy9vYmp0 b29sL2NoZWNrLmMKKysrIGIvdG9vbHMvb2JqdG9vbC9jaGVjay5jCkBAIC0z ODIwLDkgKzM4MjAsMTIgQEAgc3RhdGljIGludCB2YWxpZGF0ZV9icmFuY2go c3RydWN0IG9ianRvb2xfZmlsZSAqZmlsZSwgc3RydWN0IHN5bWJvbCAqZnVu YywKIAkJCWJyZWFrOwogCiAJCWNhc2UgSU5TTl9DT05URVhUX1NXSVRDSDoK LQkJCWlmIChmdW5jICYmICghbmV4dF9pbnNuIHx8ICFuZXh0X2luc24tPmhp bnQpKSB7Ci0JCQkJV0FSTl9JTlNOKGluc24sICJ1bnN1cHBvcnRlZCBpbnN0 cnVjdGlvbiBpbiBjYWxsYWJsZSBmdW5jdGlvbiIpOwotCQkJCXJldHVybiAx OworCQkJaWYgKGZ1bmMpIHsKKwkJCQlpZiAoIW5leHRfaW5zbiB8fCAhbmV4 dF9pbnNuLT5oaW50KSB7CisJCQkJCVdBUk5fSU5TTihpbnNuLCAidW5zdXBw b3J0ZWQgaW5zdHJ1Y3Rpb24gaW4gY2FsbGFibGUgZnVuY3Rpb24iKTsKKwkJ CQkJcmV0dXJuIDE7CisJCQkJfQorCQkJCWJyZWFrOwogCQkJfQogCQkJcmV0 dXJuIDA7CiAKLS0gCjIuNDMuMAoK --=separator Content-Type: application/octet-stream; name="xsa466-linux-03.patch" Content-Disposition: attachment; filename="xsa466-linux-03.patch" Content-Transfer-Encoding: base64 RnJvbSAwZWY4MDQ3YjczN2Q3NDgwYTVkNGM0NmQ5NTZlOTdjMTkwZjEzMDUw IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IEZyaSwgMjkgTm92IDIwMjQgMTY6 MTU6NTQgKzAxMDAKU3ViamVjdDogeDg2L3N0YXRpYy1jYWxsOiBwcm92aWRl IGEgd2F5IHRvIGRvIHZlcnkgZWFybHkKIHN0YXRpYy1jYWxsIHVwZGF0ZXMK CkFkZCBzdGF0aWNfY2FsbF91cGRhdGVfZWFybHkoKSBmb3IgdXBkYXRpbmcg c3RhdGljLWNhbGwgdGFyZ2V0cyBpbgp2ZXJ5IGVhcmx5IGJvb3QuCgpUaGlz IHdpbGwgYmUgbmVlZGVkIGZvciBzdXBwb3J0IG9mIFhlbiBndWVzdCB0eXBl IHNwZWNpZmljIGh5cGVyY2FsbApmdW5jdGlvbnMuCgpUaGlzIGlzIHBhcnQg b2YgWFNBLTQ2NiAvIENWRS0yMDI0LTUzMjQxLgoKUmVwb3J0ZWQtYnk6IEFu ZHJldyBDb29wZXIgPGFuZHJldy5jb29wZXIzQGNpdHJpeC5jb20+ClNpZ25l ZC1vZmYtYnk6IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT4KQ28t ZGV2ZWxvcGVkLWJ5OiBQZXRlciBaaWpsc3RyYSA8cGV0ZXJ6QGluZnJhZGVh ZC5vcmc+CkNvLWRldmVsb3BlZC1ieTogSm9zaCBQb2ltYm9ldWYgPGpwb2lt Ym9lQHJlZGhhdC5jb20+Ci0tLQogYXJjaC94ODYvaW5jbHVkZS9hc20vc3Rh dGljX2NhbGwuaCB8IDE1ICsrKysrKysrKysrKwogYXJjaC94ODYvaW5jbHVk ZS9hc20vc3luY19jb3JlLmggICB8ICA2ICsrLS0tCiBhcmNoL3g4Ni9rZXJu ZWwvc3RhdGljX2NhbGwuYyAgICAgIHwgIDkgKysrKysrKysKIGluY2x1ZGUv bGludXgvY29tcGlsZXIuaCAgICAgICAgICAgfCAzNyArKysrKysrKysrKysr KysrKysrKystLS0tLS0tLS0KIGluY2x1ZGUvbGludXgvc3RhdGljX2NhbGwu aCAgICAgICAgfCAgMSArCiBrZXJuZWwvc3RhdGljX2NhbGxfaW5saW5lLmMg ICAgICAgIHwgIDIgKy0KIDYgZmlsZXMgY2hhbmdlZCwgNTUgaW5zZXJ0aW9u cygrKSwgMTUgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvYXJjaC94ODYv aW5jbHVkZS9hc20vc3RhdGljX2NhbGwuaCBiL2FyY2gveDg2L2luY2x1ZGUv YXNtL3N0YXRpY19jYWxsLmgKaW5kZXggMTI1YzQwN2UyYWJlLi40MTUwMmJk MmFmZDYgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2L2luY2x1ZGUvYXNtL3N0YXRp Y19jYWxsLmgKKysrIGIvYXJjaC94ODYvaW5jbHVkZS9hc20vc3RhdGljX2Nh bGwuaApAQCAtNjUsNCArNjUsMTkgQEAKIAogZXh0ZXJuIGJvb2wgX19zdGF0 aWNfY2FsbF9maXh1cCh2b2lkICp0cmFtcCwgdTggb3AsIHZvaWQgKmRlc3Qp OwogCitleHRlcm4gdm9pZCBfX3N0YXRpY19jYWxsX3VwZGF0ZV9lYXJseSh2 b2lkICp0cmFtcCwgdm9pZCAqZnVuYyk7CisKKyNkZWZpbmUgc3RhdGljX2Nh bGxfdXBkYXRlX2Vhcmx5KG5hbWUsIF9mdW5jKQkJCQlcCisoewkJCQkJCQkJ CVwKKwl0eXBlb2YoJlNUQVRJQ19DQUxMX1RSQU1QKG5hbWUpKSBfX0YgPSAo X2Z1bmMpOwkJCVwKKwlpZiAoc3RhdGljX2NhbGxfaW5pdGlhbGl6ZWQpIHsJ CQkJCVwKKwkJX19zdGF0aWNfY2FsbF91cGRhdGUoJlNUQVRJQ19DQUxMX0tF WShuYW1lKSwJCVwKKwkJCQkgICAgIFNUQVRJQ19DQUxMX1RSQU1QX0FERFIo bmFtZSksIF9fRik7XAorCX0gZWxzZSB7CQkJCQkJCVwKKwkJV1JJVEVfT05D RShTVEFUSUNfQ0FMTF9LRVkobmFtZSkuZnVuYywgX2Z1bmMpOwkJXAorCQlf X3N0YXRpY19jYWxsX3VwZGF0ZV9lYXJseShTVEFUSUNfQ0FMTF9UUkFNUF9B RERSKG5hbWUpLFwKKwkJCQkJICAgX19GKTsJCQlcCisJfQkJCQkJCQkJXAor fSkKKwogI2VuZGlmIC8qIF9BU01fU1RBVElDX0NBTExfSCAqLwpkaWZmIC0t Z2l0IGEvYXJjaC94ODYvaW5jbHVkZS9hc20vc3luY19jb3JlLmggYi9hcmNo L3g4Ni9pbmNsdWRlL2FzbS9zeW5jX2NvcmUuaAppbmRleCBhYjczODJmOTJh ZmYuLjk2YmRhNDM1MzhlZSAxMDA2NDQKLS0tIGEvYXJjaC94ODYvaW5jbHVk ZS9hc20vc3luY19jb3JlLmgKKysrIGIvYXJjaC94ODYvaW5jbHVkZS9hc20v c3luY19jb3JlLmgKQEAgLTgsNyArOCw3IEBACiAjaW5jbHVkZSA8YXNtL3Nw ZWNpYWxfaW5zbnMuaD4KIAogI2lmZGVmIENPTkZJR19YODZfMzIKLXN0YXRp YyBpbmxpbmUgdm9pZCBpcmV0X3RvX3NlbGYodm9pZCkKK3N0YXRpYyBfX2Fs d2F5c19pbmxpbmUgdm9pZCBpcmV0X3RvX3NlbGYodm9pZCkKIHsKIAlhc20g dm9sYXRpbGUgKAogCQkicHVzaGZsXG5cdCIKQEAgLTE5LDcgKzE5LDcgQEAg c3RhdGljIGlubGluZSB2b2lkIGlyZXRfdG9fc2VsZih2b2lkKQogCQk6IEFT TV9DQUxMX0NPTlNUUkFJTlQgOiA6ICJtZW1vcnkiKTsKIH0KICNlbHNlCi1z dGF0aWMgaW5saW5lIHZvaWQgaXJldF90b19zZWxmKHZvaWQpCitzdGF0aWMg X19hbHdheXNfaW5saW5lIHZvaWQgaXJldF90b19zZWxmKHZvaWQpCiB7CiAJ dW5zaWduZWQgaW50IHRtcDsKIApAQCAtNTUsNyArNTUsNyBAQCBzdGF0aWMg aW5saW5lIHZvaWQgaXJldF90b19zZWxmKHZvaWQpCiAgKiBMaWtlIGFsbCBv ZiBMaW51eCdzIG1lbW9yeSBvcmRlcmluZyBvcGVyYXRpb25zLCB0aGlzIGlz IGEKICAqIGNvbXBpbGVyIGJhcnJpZXIgYXMgd2VsbC4KICAqLwotc3RhdGlj IGlubGluZSB2b2lkIHN5bmNfY29yZSh2b2lkKQorc3RhdGljIF9fYWx3YXlz X2lubGluZSB2b2lkIHN5bmNfY29yZSh2b2lkKQogewogCS8qCiAJICogVGhl IFNFUklBTElaRSBpbnN0cnVjdGlvbiBpcyB0aGUgbW9zdCBzdHJhaWdodGZv cndhcmQgd2F5IHRvCmRpZmYgLS1naXQgYS9hcmNoL3g4Ni9rZXJuZWwvc3Rh dGljX2NhbGwuYyBiL2FyY2gveDg2L2tlcm5lbC9zdGF0aWNfY2FsbC5jCmlu ZGV4IDRlZWZhYWM2NGM2Yy4uOWVlZDBjMTQ0ZGFkIDEwMDY0NAotLS0gYS9h cmNoL3g4Ni9rZXJuZWwvc3RhdGljX2NhbGwuYworKysgYi9hcmNoL3g4Ni9r ZXJuZWwvc3RhdGljX2NhbGwuYwpAQCAtMTcyLDYgKzE3MiwxNSBAQCB2b2lk IGFyY2hfc3RhdGljX2NhbGxfdHJhbnNmb3JtKHZvaWQgKnNpdGUsIHZvaWQg KnRyYW1wLCB2b2lkICpmdW5jLCBib29sIHRhaWwpCiB9CiBFWFBPUlRfU1lN Qk9MX0dQTChhcmNoX3N0YXRpY19jYWxsX3RyYW5zZm9ybSk7CiAKK25vaW5z dHIgdm9pZCBfX3N0YXRpY19jYWxsX3VwZGF0ZV9lYXJseSh2b2lkICp0cmFt cCwgdm9pZCAqZnVuYykKK3sKKwlCVUdfT04oc3lzdGVtX3N0YXRlICE9IFNZ U1RFTV9CT09USU5HKTsKKwlCVUdfT04oIWVhcmx5X2Jvb3RfaXJxc19kaXNh YmxlZCk7CisJQlVHX09OKHN0YXRpY19jYWxsX2luaXRpYWxpemVkKTsKKwlf X3RleHRfZ2VuX2luc24odHJhbXAsIEpNUDMyX0lOU05fT1BDT0RFLCB0cmFt cCwgZnVuYywgSk1QMzJfSU5TTl9TSVpFKTsKKwlzeW5jX2NvcmUoKTsKK30K KwogI2lmZGVmIENPTkZJR19NSVRJR0FUSU9OX1JFVEhVTksKIC8qCiAgKiBU aGlzIGlzIGNhbGxlZCBieSBhcHBseV9yZXR1cm5zKCkgdG8gZml4IHVwIHN0 YXRpYyBjYWxsIHRyYW1wb2xpbmVzLApkaWZmIC0tZ2l0IGEvaW5jbHVkZS9s aW51eC9jb21waWxlci5oIGIvaW5jbHVkZS9saW51eC9jb21waWxlci5oCmlu ZGV4IDQ2OWE2NGRkNjQ5NS4uMjQwYzYzMmM1Yjk1IDEwMDY0NAotLS0gYS9p bmNsdWRlL2xpbnV4L2NvbXBpbGVyLmgKKysrIGIvaW5jbHVkZS9saW51eC9j b21waWxlci5oCkBAIC0yMTYsMjggKzIxNiw0MyBAQCB2b2lkIGZ0cmFjZV9s aWtlbHlfdXBkYXRlKHN0cnVjdCBmdHJhY2VfbGlrZWx5X2RhdGEgKmYsIGlu dCB2YWwsCiAKICNlbmRpZiAvKiBfX0tFUk5FTF9fICovCiAKKy8qKgorICog b2Zmc2V0X3RvX3B0ciAtIGNvbnZlcnQgYSByZWxhdGl2ZSBtZW1vcnkgb2Zm c2V0IHRvIGFuIGFic29sdXRlIHBvaW50ZXIKKyAqIEBvZmY6CXRoZSBhZGRy ZXNzIG9mIHRoZSAzMi1iaXQgb2Zmc2V0IHZhbHVlCisgKi8KK3N0YXRpYyBp bmxpbmUgdm9pZCAqb2Zmc2V0X3RvX3B0cihjb25zdCBpbnQgKm9mZikKK3sK KwlyZXR1cm4gKHZvaWQgKikoKHVuc2lnbmVkIGxvbmcpb2ZmICsgKm9mZik7 Cit9CisKKyNlbmRpZiAvKiBfX0FTU0VNQkxZX18gKi8KKworI2lmZGVmIENP TkZJR182NEJJVAorI2RlZmluZSBBUkNIX1NFTChhLGIpIGEKKyNlbHNlCisj ZGVmaW5lIEFSQ0hfU0VMKGEsYikgYgorI2VuZGlmCisKIC8qCiAgKiBGb3Jj ZSB0aGUgY29tcGlsZXIgdG8gZW1pdCAnc3ltJyBhcyBhIHN5bWJvbCwgc28g dGhhdCB3ZSBjYW4gcmVmZXJlbmNlCiAgKiBpdCBmcm9tIGlubGluZSBhc3Nl bWJsZXIuIE5lY2Vzc2FyeSBpbiBjYXNlICdzeW0nIGNvdWxkIGJlIGlubGlu ZWQKICAqIG90aGVyd2lzZSwgb3IgZWxpbWluYXRlZCBlbnRpcmVseSBkdWUg dG8gbGFjayBvZiByZWZlcmVuY2VzIHRoYXQgYXJlCiAgKiB2aXNpYmxlIHRv IHRoZSBjb21waWxlci4KICAqLwotI2RlZmluZSBfX19BRERSRVNTQUJMRShz eW0sIF9fYXR0cnMpIFwKLQlzdGF0aWMgdm9pZCAqIF9fdXNlZCBfX2F0dHJz IFwKKyNkZWZpbmUgX19fQUREUkVTU0FCTEUoc3ltLCBfX2F0dHJzKQkJCQkJ CVwKKwlzdGF0aWMgdm9pZCAqIF9fdXNlZCBfX2F0dHJzCQkJCQkJXAogCV9f VU5JUVVFX0lEKF9fUEFTVEUoX19hZGRyZXNzYWJsZV8sc3ltKSkgPSAodm9p ZCAqKSh1aW50cHRyX3QpJnN5bTsKKwogI2RlZmluZSBfX0FERFJFU1NBQkxF KHN5bSkgXAogCV9fX0FERFJFU1NBQkxFKHN5bSwgX19zZWN0aW9uKCIuZGlz Y2FyZC5hZGRyZXNzYWJsZSIpKQogCi0vKioKLSAqIG9mZnNldF90b19wdHIg LSBjb252ZXJ0IGEgcmVsYXRpdmUgbWVtb3J5IG9mZnNldCB0byBhbiBhYnNv bHV0ZSBwb2ludGVyCi0gKiBAb2ZmOgl0aGUgYWRkcmVzcyBvZiB0aGUgMzIt Yml0IG9mZnNldCB2YWx1ZQotICovCi1zdGF0aWMgaW5saW5lIHZvaWQgKm9m ZnNldF90b19wdHIoY29uc3QgaW50ICpvZmYpCi17Ci0JcmV0dXJuICh2b2lk ICopKCh1bnNpZ25lZCBsb25nKW9mZiArICpvZmYpOwotfQorI2RlZmluZSBf X0FERFJFU1NBQkxFX0FTTShzeW0pCQkJCQkJXAorCS5wdXNoc2VjdGlvbiAu ZGlzY2FyZC5hZGRyZXNzYWJsZSwiYXciOwkJCQlcCisJLmFsaWduIEFSQ0hf U0VMKDgsNCk7CQkJCQkJXAorCUFSQ0hfU0VMKC5xdWFkLCAubG9uZykgX19z dHJpbmdpZnkoc3ltKTsJCQlcCisJLnBvcHNlY3Rpb247CiAKLSNlbmRpZiAv KiBfX0FTU0VNQkxZX18gKi8KKyNkZWZpbmUgX19BRERSRVNTQUJMRV9BU01f U1RSKHN5bSkgX19zdHJpbmdpZnkoX19BRERSRVNTQUJMRV9BU00oc3ltKSkK IAogI2lmZGVmIF9fQ0hFQ0tFUl9fCiAjZGVmaW5lIF9fQlVJTERfQlVHX09O X1pFUk9fTVNHKGUsIG1zZykgKDApCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xp bnV4L3N0YXRpY19jYWxsLmggYi9pbmNsdWRlL2xpbnV4L3N0YXRpY19jYWxs LmgKaW5kZXggMTQxZTZiMTc2YTFiLi43ODU5ODBhZjg5NzIgMTAwNjQ0Ci0t LSBhL2luY2x1ZGUvbGludXgvc3RhdGljX2NhbGwuaAorKysgYi9pbmNsdWRl L2xpbnV4L3N0YXRpY19jYWxsLmgKQEAgLTEzOCw2ICsxMzgsNyBAQAogI2lm ZGVmIENPTkZJR19IQVZFX1NUQVRJQ19DQUxMCiAjaW5jbHVkZSA8YXNtL3N0 YXRpY19jYWxsLmg+CiAKK2V4dGVybiBpbnQgc3RhdGljX2NhbGxfaW5pdGlh bGl6ZWQ7CiAvKgogICogRWl0aGVyIEBzaXRlIG9yIEB0cmFtcCBjYW4gYmUg TlVMTC4KICAqLwpkaWZmIC0tZ2l0IGEva2VybmVsL3N0YXRpY19jYWxsX2lu bGluZS5jIGIva2VybmVsL3N0YXRpY19jYWxsX2lubGluZS5jCmluZGV4IDUy NTljZGE0ODZkMC4uYmI3ZDA2NmE3YzM5IDEwMDY0NAotLS0gYS9rZXJuZWwv c3RhdGljX2NhbGxfaW5saW5lLmMKKysrIGIva2VybmVsL3N0YXRpY19jYWxs X2lubGluZS5jCkBAIC0xNSw3ICsxNSw3IEBAIGV4dGVybiBzdHJ1Y3Qgc3Rh dGljX2NhbGxfc2l0ZSBfX3N0YXJ0X3N0YXRpY19jYWxsX3NpdGVzW10sCiBl eHRlcm4gc3RydWN0IHN0YXRpY19jYWxsX3RyYW1wX2tleSBfX3N0YXJ0X3N0 YXRpY19jYWxsX3RyYW1wX2tleVtdLAogCQkJCSAgICBfX3N0b3Bfc3RhdGlj X2NhbGxfdHJhbXBfa2V5W107CiAKLXN0YXRpYyBpbnQgc3RhdGljX2NhbGxf aW5pdGlhbGl6ZWQ7CitpbnQgc3RhdGljX2NhbGxfaW5pdGlhbGl6ZWQ7CiAK IC8qCiAgKiBNdXN0IGJlIGNhbGxlZCBiZWZvcmUgZWFybHlfaW5pdGNhbGwo KSB0byBiZSBlZmZlY3RpdmUuCi0tIAoyLjQzLjAKCg== --=separator Content-Type: application/octet-stream; name="xsa466-linux-04.patch" Content-Disposition: attachment; filename="xsa466-linux-04.patch" Content-Transfer-Encoding: base64 RnJvbSBhMjc5NmRmZjYyZDZjNmJmYzVmYmViZGYyYmVlMGQ1YWMwNDM4OTA2 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IFdlZCwgMTYgT2N0IDIwMjQgMTA6 NDA6MjYgKzAyMDAKU3ViamVjdDogeDg2L3hlbjogZG9uJ3QgZG8gUFYgaXJl dCBoeXBlcmNhbGwgdGhyb3VnaCBoeXBlcmNhbGwKIHBhZ2UKCkluc3RlYWQg b2YganVtcGluZyB0byB0aGUgWGVuIGh5cGVyY2FsbCBwYWdlIGZvciBkb2lu ZyB0aGUgaXJldApoeXBlcmNhbGwsIGRpcmVjdGx5IGNvZGUgdGhlIHJlcXVp cmVkIHNlcXVlbmNlIGluIHhlbi1hc20uUy4KClRoaXMgaXMgZG9uZSBpbiBw cmVwYXJhdGlvbiBvZiBubyBsb25nZXIgdXNpbmcgaHlwZXJjYWxsIHBhZ2Ug YXQgYWxsLAphcyBpdCBoYXMgc2hvd24gdG8gY2F1c2UgcHJvYmxlbXMgd2l0 aCBzcGVjdWxhdGlvbiBtaXRpZ2F0aW9ucy4KClRoaXMgaXMgcGFydCBvZiBY U0EtNDY2IC8gQ1ZFLTIwMjQtNTMyNDEuCgpSZXBvcnRlZC1ieTogQW5kcmV3 IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KU2lnbmVkLW9m Zi1ieTogSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPgpSZXZpZXdl ZC1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgotLS0KIGFy Y2gveDg2L3hlbi94ZW4tYXNtLlMgfCAyNyArKysrKysrKysrKysrKysrKyst LS0tLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAxOCBpbnNlcnRpb25zKCspLCA5 IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2FyY2gveDg2L3hlbi94ZW4t YXNtLlMgYi9hcmNoL3g4Ni94ZW4veGVuLWFzbS5TCmluZGV4IDgzMTg5Y2Y1 Y2RjZS4uY2E2ZWRmZTRjMTRiIDEwMDY0NAotLS0gYS9hcmNoL3g4Ni94ZW4v eGVuLWFzbS5TCisrKyBiL2FyY2gveDg2L3hlbi94ZW4tYXNtLlMKQEAgLTE3 Niw3ICsxNzYsNiBAQCBTWU1fQ09ERV9TVEFSVCh4ZW5fZWFybHlfaWR0X2hh bmRsZXJfYXJyYXkpCiBTWU1fQ09ERV9FTkQoeGVuX2Vhcmx5X2lkdF9oYW5k bGVyX2FycmF5KQogCV9fRklOSVQKIAotaHlwZXJjYWxsX2lyZXQgPSBoeXBl cmNhbGxfcGFnZSArIF9fSFlQRVJWSVNPUl9pcmV0ICogMzIKIC8qCiAgKiBY ZW42NCBpcmV0IGZyYW1lOgogICoKQEAgLTE4NiwxNyArMTg1LDI4IEBAIGh5 cGVyY2FsbF9pcmV0ID0gaHlwZXJjYWxsX3BhZ2UgKyBfX0hZUEVSVklTT1Jf aXJldCAqIDMyCiAgKgljcwogICoJcmlwCQk8LS0gc3RhbmRhcmQgaXJldCBm cmFtZQogICoKLSAqCWZsYWdzCisgKglmbGFncwkJPC0tIHhlbl9pcmV0IG11 c3QgcHVzaCBmcm9tIGhlcmUgb24KICAqCi0gKglyY3gJCX0KLSAqCXIxMQkJ fTwtLSBwdXNoZWQgYnkgaHlwZXJjYWxsIHBhZ2UKLSAqIHJzcC0+cmF4CQl9 CisgKglyY3gKKyAqCXIxMQorICogcnNwLT5yYXgKICAqLworLm1hY3JvIHhl bl9oeXBlcmNhbGxfaXJldAorCXB1c2hxICQwCS8qIEZsYWdzICovCisJcHVz aCAlcmN4CisJcHVzaCAlcjExCisJcHVzaCAlcmF4CisJbW92ICAkX19IWVBF UlZJU09SX2lyZXQsICVlYXgKKwlzeXNjYWxsCQkvKiBEbyB0aGUgSVJFVC4g Ki8KKyNpZmRlZiBDT05GSUdfTUlUSUdBVElPTl9TTFMKKwlpbnQzCisjZW5k aWYKKy5lbmRtCisKIFNZTV9DT0RFX1NUQVJUKHhlbl9pcmV0KQogCVVOV0lO RF9ISU5UX1VOREVGSU5FRAogCUFOTk9UQVRFX05PRU5EQlIKLQlwdXNocSAk MAotCWptcCBoeXBlcmNhbGxfaXJldAorCXhlbl9oeXBlcmNhbGxfaXJldAog U1lNX0NPREVfRU5EKHhlbl9pcmV0KQogCiAvKgpAQCAtMzAxLDggKzMxMSw3 IEBAIFNZTV9DT0RFX1NUQVJUKHhlbl9lbnRyeV9TWVNFTlRFUl9jb21wYXQp CiAJRU5EQlIKIAlsZWEgMTYoJXJzcCksICVyc3AJLyogc3RyaXAgJXJjeCwg JXIxMSAqLwogCW1vdiAkLUVOT1NZUywgJXJheAotCXB1c2hxICQwCi0Jam1w IGh5cGVyY2FsbF9pcmV0CisJeGVuX2h5cGVyY2FsbF9pcmV0CiBTWU1fQ09E RV9FTkQoeGVuX2VudHJ5X1NZU0VOVEVSX2NvbXBhdCkKIFNZTV9DT0RFX0VO RCh4ZW5fZW50cnlfU1lTQ0FMTF9jb21wYXQpCiAKLS0gCjIuNDMuMAoK --=separator Content-Type: application/octet-stream; name="xsa466-linux-05.patch" Content-Disposition: attachment; filename="xsa466-linux-05.patch" Content-Transfer-Encoding: base64 RnJvbSBiNDg0NWJiNjM4MzgyMWE5NTE2Y2UzMGFmM2EyN2RjODczZTM3ZmQ0 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IFRodSwgMTcgT2N0IDIwMjQgMTE6 MDA6NTIgKzAyMDAKU3ViamVjdDogeDg2L3hlbjogYWRkIGNlbnRyYWwgaHlw ZXJjYWxsIGZ1bmN0aW9ucwoKQWRkIGdlbmVyaWMgaHlwZXJjYWxsIGZ1bmN0 aW9ucyB1c2FibGUgZm9yIGFsbCBub3JtYWwgKGkuZS4gbm90IGlyZXQpCmh5 cGVyY2FsbHMuIERlcGVuZGluZyBvbiB0aGUgZ3Vlc3QgdHlwZSBhbmQgdGhl IHByb2Nlc3NvciB2ZW5kb3IKZGlmZmVyZW50IGZ1bmN0aW9ucyBuZWVkIHRv IGJlIHVzZWQgZHVlIHRvIHRoZSB0byBiZSB1c2VkIGluc3RydWN0aW9uCmZv ciBlbnRlcmluZyB0aGUgaHlwZXJ2aXNvcjoKCi0gUFYgZ3Vlc3RzIG5lZWQg dG8gdXNlIHN5c2NhbGwKLSBIVk0vUFZIIGd1ZXN0cyBvbiBJbnRlbCBuZWVk IHRvIHVzZSB2bWNhbGwKLSBIVk0vUFZIIGd1ZXN0cyBvbiBBTUQgYW5kIEh5 Z29uIG5lZWQgdG8gdXNlIHZtbWNhbGwKCkFzIFBWSCBndWVzdHMgbmVlZCB0 byBpc3N1ZSBoeXBlcmNhbGxzIHZlcnkgZWFybHkgZHVyaW5nIGJvb3QsIHRo ZXJlCmlzIGEgNHRoIGh5cGVyY2FsbCBmdW5jdGlvbiBuZWVkZWQgZm9yIEhW TS9QVkggd2hpY2ggY2FuIGJlIHVzZWQgb24KSW50ZWwgYW5kIEFNRCBwcm9j ZXNzb3JzLiBJdCB3aWxsIGNoZWNrIHRoZSB2ZW5kb3IgdHlwZSBhbmQgdGhl biBzZXQKdGhlIEludGVsIG9yIEFNRCBzcGVjaWZpYyBmdW5jdGlvbiB0byB1 c2UgdmlhIHN0YXRpY19jYWxsKCkuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTQ2 NiAvIENWRS0yMDI0LTUzMjQxLgoKUmVwb3J0ZWQtYnk6IEFuZHJldyBDb29w ZXIgPGFuZHJldy5jb29wZXIzQGNpdHJpeC5jb20+ClNpZ25lZC1vZmYtYnk6 IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT4KQ28tZGV2ZWxvcGVk LWJ5OiBQZXRlciBaaWpsc3RyYSA8cGV0ZXJ6QGluZnJhZGVhZC5vcmc+Ci0t LQogYXJjaC94ODYvaW5jbHVkZS9hc20veGVuL2h5cGVyY2FsbC5oIHwgIDMg KwogYXJjaC94ODYveGVuL2VubGlnaHRlbi5jICAgICAgICAgICAgIHwgNjUg KysrKysrKysrKysrKysrKysrKysrKwogYXJjaC94ODYveGVuL2VubGlnaHRl bl9odm0uYyAgICAgICAgIHwgIDQgKysKIGFyY2gveDg2L3hlbi9lbmxpZ2h0 ZW5fcHYuYyAgICAgICAgICB8ICA0ICstCiBhcmNoL3g4Ni94ZW4veGVuLWFz bS5TICAgICAgICAgICAgICAgfCAyMyArKysrKysrKwogYXJjaC94ODYveGVu L3hlbi1oZWFkLlMgICAgICAgICAgICAgIHwgODMgKysrKysrKysrKysrKysr KysrKysrKysrKysrKwogYXJjaC94ODYveGVuL3hlbi1vcHMuaCAgICAgICAg ICAgICAgIHwgIDkgKysrCiA3IGZpbGVzIGNoYW5nZWQsIDE5MCBpbnNlcnRp b25zKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvYXJjaC94ODYv aW5jbHVkZS9hc20veGVuL2h5cGVyY2FsbC5oIGIvYXJjaC94ODYvaW5jbHVk ZS9hc20veGVuL2h5cGVyY2FsbC5oCmluZGV4IGEyZGQyNDk0N2ViOC4uNmI0 ZGQ0ZGUwOGE2IDEwMDY0NAotLS0gYS9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94 ZW4vaHlwZXJjYWxsLmgKKysrIGIvYXJjaC94ODYvaW5jbHVkZS9hc20veGVu L2h5cGVyY2FsbC5oCkBAIC04OCw2ICs4OCw5IEBAIHN0cnVjdCB4ZW5fZG1f b3BfYnVmOwogCiBleHRlcm4gc3RydWN0IHsgY2hhciBfZW50cnlbMzJdOyB9 IGh5cGVyY2FsbF9wYWdlW107CiAKK3ZvaWQgeGVuX2h5cGVyY2FsbF9mdW5j KHZvaWQpOworREVDTEFSRV9TVEFUSUNfQ0FMTCh4ZW5faHlwZXJjYWxsLCB4 ZW5faHlwZXJjYWxsX2Z1bmMpOworCiAjZGVmaW5lIF9fSFlQRVJDQUxMCQki Y2FsbCBoeXBlcmNhbGxfcGFnZSslY1tvZmZzZXRdIgogI2RlZmluZSBfX0hZ UEVSQ0FMTF9FTlRSWSh4KQkJCQkJCVwKIAlbb2Zmc2V0XSAiaSIgKF9fSFlQ RVJWSVNPUl8jI3ggKiBzaXplb2YoaHlwZXJjYWxsX3BhZ2VbMF0pKQpkaWZm IC0tZ2l0IGEvYXJjaC94ODYveGVuL2VubGlnaHRlbi5jIGIvYXJjaC94ODYv eGVuL2VubGlnaHRlbi5jCmluZGV4IDg0ZTVhZGJkMDkyNS4uMTg4NzQzNWFm MmZiIDEwMDY0NAotLS0gYS9hcmNoL3g4Ni94ZW4vZW5saWdodGVuLmMKKysr IGIvYXJjaC94ODYveGVuL2VubGlnaHRlbi5jCkBAIC0yLDYgKzIsNyBAQAog CiAjaW5jbHVkZSA8bGludXgvY29uc29sZS5oPgogI2luY2x1ZGUgPGxpbnV4 L2NwdS5oPgorI2luY2x1ZGUgPGxpbnV4L2luc3RydW1lbnRhdGlvbi5oPgog I2luY2x1ZGUgPGxpbnV4L2tleGVjLmg+CiAjaW5jbHVkZSA8bGludXgvbWVt YmxvY2suaD4KICNpbmNsdWRlIDxsaW51eC9zbGFiLmg+CkBAIC0yMyw2ICsy NCw5IEBACiAKIEVYUE9SVF9TWU1CT0xfR1BMKGh5cGVyY2FsbF9wYWdlKTsK IAorREVGSU5FX1NUQVRJQ19DQUxMKHhlbl9oeXBlcmNhbGwsIHhlbl9oeXBl cmNhbGxfaHZtKTsKK0VYUE9SVF9TVEFUSUNfQ0FMTF9UUkFNUCh4ZW5faHlw ZXJjYWxsKTsKKwogLyoKICAqIFBvaW50ZXIgdG8gdGhlIHhlbl92Y3B1X2lu Zm8gc3RydWN0dXJlIG9yCiAgKiAmSFlQRVJWSVNPUl9zaGFyZWRfaW5mby0+ dmNwdV9pbmZvW2NwdV0uIFNlZSB4ZW5faHZtX2luaXRfc2hhcmVkX2luZm8K QEAgLTY4LDYgKzcyLDY3IEBAIEVYUE9SVF9TWU1CT0woeGVuX3N0YXJ0X2Zs YWdzKTsKICAqLwogc3RydWN0IHNoYXJlZF9pbmZvICpIWVBFUlZJU09SX3No YXJlZF9pbmZvID0gJnhlbl9kdW1teV9zaGFyZWRfaW5mbzsKIAorc3RhdGlj IF9fcmVmIHZvaWQgeGVuX2dldF92ZW5kb3Iodm9pZCkKK3sKKwlpbml0X2Nw dV9kZXZzKCk7CisJY3B1X2RldGVjdCgmYm9vdF9jcHVfZGF0YSk7CisJZ2V0 X2NwdV92ZW5kb3IoJmJvb3RfY3B1X2RhdGEpOworfQorCit2b2lkIHhlbl9o eXBlcmNhbGxfc2V0ZnVuYyh2b2lkKQoreworCWlmIChzdGF0aWNfY2FsbF9x dWVyeSh4ZW5faHlwZXJjYWxsKSAhPSB4ZW5faHlwZXJjYWxsX2h2bSkKKwkJ cmV0dXJuOworCisJaWYgKChib290X2NwdV9kYXRhLng4Nl92ZW5kb3IgPT0g WDg2X1ZFTkRPUl9BTUQgfHwKKwkgICAgIGJvb3RfY3B1X2RhdGEueDg2X3Zl bmRvciA9PSBYODZfVkVORE9SX0hZR09OKSkKKwkJc3RhdGljX2NhbGxfdXBk YXRlKHhlbl9oeXBlcmNhbGwsIHhlbl9oeXBlcmNhbGxfYW1kKTsKKwllbHNl CisJCXN0YXRpY19jYWxsX3VwZGF0ZSh4ZW5faHlwZXJjYWxsLCB4ZW5faHlw ZXJjYWxsX2ludGVsKTsKK30KKworLyoKKyAqIEV2YWx1YXRlIHByb2Nlc3Nv ciB2ZW5kb3IgaW4gb3JkZXIgdG8gc2VsZWN0IHRoZSBjb3JyZWN0IGh5cGVy Y2FsbAorICogZnVuY3Rpb24gZm9yIEhWTS9QVkggZ3Vlc3RzLgorICogTWln aHQgYmUgY2FsbGVkIHZlcnkgZWFybHkgaW4gYm9vdCBiZWZvcmUgdmVuZG9y IGhhcyBiZWVuIHNldCBieQorICogZWFybHlfY3B1X2luaXQoKS4KKyAqLwor bm9pbnN0ciB2b2lkICpfX3hlbl9oeXBlcmNhbGxfc2V0ZnVuYyh2b2lkKQor eworCXZvaWQgKCpmdW5jKSh2b2lkKTsKKworCS8qCisJICogWGVuIGlzIHN1 cHBvcnRlZCBvbmx5IG9uIENQVXMgd2l0aCBDUFVJRCwgc28gdGVzdGluZyBm b3IKKwkgKiBYODZfRkVBVFVSRV9DUFVJRCBpcyBhIHRlc3QgZm9yIGVhcmx5 X2NwdV9pbml0KCkgaGF2aW5nIGJlZW4KKwkgKiBydW4uCisJICoKKwkgKiBO b3RlIHRoYXQgX194ZW5faHlwZXJjYWxsX3NldGZ1bmMoKSBpcyBub2luc3Ry IG9ubHkgZHVlIHRvIGEgbmFzdHkKKwkgKiBkZXBlbmRlbmN5IGNoYWluOiBp dCBpcyBiZWluZyBjYWxsZWQgdmlhIHRoZSB4ZW5faHlwZXJjYWxsIHN0YXRp YworCSAqIGNhbGwgd2hlbiBydW5uaW5nIGFzIGEgUFZIIG9yIEhWTSBndWVz dC4gSHlwZXJjYWxscyBuZWVkIHRvIGJlCisJICogbm9pbnN0ciBkdWUgdG8g UFYgZ3Vlc3RzIHVzaW5nIGh5cGVyY2FsbHMgaW4gbm9pbnN0ciBjb2RlLiBT byB3ZQorCSAqIGNhbiBzYWZlbHkgdGFnIHRoZSBmdW5jdGlvbiBib2R5IGFz ICJpbnN0cnVtZW50YXRpb24gb2siLCBzaW5jZQorCSAqIHRoZSBQViBndWVz dCByZXF1aXJlbWVudCBpcyBub3Qgb2YgaW50ZXJlc3QgaGVyZSAoeGVuX2dl dF92ZW5kb3IoKQorCSAqIGNhbGxzIG5vaW5zdHIgZnVuY3Rpb25zLCBhbmQg c3RhdGljX2NhbGxfdXBkYXRlX2Vhcmx5KCkgbWlnaHQgZG8KKwkgKiBzbywg dG9vKS4KKwkgKi8KKwlpbnN0cnVtZW50YXRpb25fYmVnaW4oKTsKKworCWlm ICghYm9vdF9jcHVfaGFzKFg4Nl9GRUFUVVJFX0NQVUlEKSkKKwkJeGVuX2dl dF92ZW5kb3IoKTsKKworCWlmICgoYm9vdF9jcHVfZGF0YS54ODZfdmVuZG9y ID09IFg4Nl9WRU5ET1JfQU1EIHx8CisJICAgICBib290X2NwdV9kYXRhLng4 Nl92ZW5kb3IgPT0gWDg2X1ZFTkRPUl9IWUdPTikpCisJCWZ1bmMgPSB4ZW5f aHlwZXJjYWxsX2FtZDsKKwllbHNlCisJCWZ1bmMgPSB4ZW5faHlwZXJjYWxs X2ludGVsOworCisJc3RhdGljX2NhbGxfdXBkYXRlX2Vhcmx5KHhlbl9oeXBl cmNhbGwsIGZ1bmMpOworCisJaW5zdHJ1bWVudGF0aW9uX2VuZCgpOworCisJ cmV0dXJuIGZ1bmM7Cit9CisKIHN0YXRpYyBpbnQgeGVuX2NwdV91cF9vbmxp bmUodW5zaWduZWQgaW50IGNwdSkKIHsKIAl4ZW5faW5pdF9sb2NrX2NwdShj cHUpOwpkaWZmIC0tZ2l0IGEvYXJjaC94ODYveGVuL2VubGlnaHRlbl9odm0u YyBiL2FyY2gveDg2L3hlbi9lbmxpZ2h0ZW5faHZtLmMKaW5kZXggMjRkMjk1 N2E0NzI2Li45NzNhNzRmYzk2NmEgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2L3hl bi9lbmxpZ2h0ZW5faHZtLmMKKysrIGIvYXJjaC94ODYveGVuL2VubGlnaHRl bl9odm0uYwpAQCAtMzAwLDYgKzMwMCwxMCBAQCBzdGF0aWMgdWludDMyX3Qg X19pbml0IHhlbl9wbGF0Zm9ybV9odm0odm9pZCkKIAlpZiAoeGVuX3B2X2Rv bWFpbigpKQogCQlyZXR1cm4gMDsKIAorCS8qIFNldCBjb3JyZWN0IGh5cGVy Y2FsbCBmdW5jdGlvbi4gKi8KKwlpZiAoeGVuX2RvbWFpbikKKwkJeGVuX2h5 cGVyY2FsbF9zZXRmdW5jKCk7CisKIAlpZiAoeGVuX3B2aF9kb21haW4oKSAm JiBub3B2KSB7CiAJCS8qIEd1ZXN0IGJvb3RpbmcgdmlhIHRoZSBYZW4tUFZI IGJvb3QgZW50cnkgZ29lcyBoZXJlICovCiAJCXByX2luZm8oIlwibm9wdlwi IHBhcmFtZXRlciBpcyBpZ25vcmVkIGluIFBWSCBndWVzdFxuIik7CmRpZmYg LS1naXQgYS9hcmNoL3g4Ni94ZW4vZW5saWdodGVuX3B2LmMgYi9hcmNoL3g4 Ni94ZW4vZW5saWdodGVuX3B2LmMKaW5kZXggZDY4MThjNmNhZmRhLi5hOGVi N2UwYzQ3M2MgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2L3hlbi9lbmxpZ2h0ZW5f cHYuYworKysgYi9hcmNoL3g4Ni94ZW4vZW5saWdodGVuX3B2LmMKQEAgLTEz NDEsNiArMTM0MSw5IEBAIGFzbWxpbmthZ2UgX192aXNpYmxlIHZvaWQgX19p bml0IHhlbl9zdGFydF9rZXJuZWwoc3RydWN0IHN0YXJ0X2luZm8gKnNpKQog CiAJeGVuX2RvbWFpbl90eXBlID0gWEVOX1BWX0RPTUFJTjsKIAl4ZW5fc3Rh cnRfZmxhZ3MgPSB4ZW5fc3RhcnRfaW5mby0+ZmxhZ3M7CisJLyogSW50ZXJy dXB0cyBhcmUgZ3VhcmFudGVlZCB0byBiZSBvZmYgaW5pdGlhbGx5LiAqLwor CWVhcmx5X2Jvb3RfaXJxc19kaXNhYmxlZCA9IHRydWU7CisJc3RhdGljX2Nh bGxfdXBkYXRlX2Vhcmx5KHhlbl9oeXBlcmNhbGwsIHhlbl9oeXBlcmNhbGxf cHYpOwogCiAJeGVuX3NldHVwX2ZlYXR1cmVzKCk7CiAKQEAgLTE0MzEsNyAr MTQzNCw2IEBAIGFzbWxpbmthZ2UgX192aXNpYmxlIHZvaWQgX19pbml0IHhl bl9zdGFydF9rZXJuZWwoc3RydWN0IHN0YXJ0X2luZm8gKnNpKQogCVdBUk5f T04oeGVuX2NwdWhwX3NldHVwKHhlbl9jcHVfdXBfcHJlcGFyZV9wdiwgeGVu X2NwdV9kZWFkX3B2KSk7CiAKIAlsb2NhbF9pcnFfZGlzYWJsZSgpOwotCWVh cmx5X2Jvb3RfaXJxc19kaXNhYmxlZCA9IHRydWU7CiAKIAl4ZW5fcmF3X2Nv bnNvbGVfd3JpdGUoIm1hcHBpbmcga2VybmVsIGludG8gcGh5c2ljYWwgbWVt b3J5XG4iKTsKIAl4ZW5fc2V0dXBfa2VybmVsX3BhZ2V0YWJsZSgocGdkX3Qg Kil4ZW5fc3RhcnRfaW5mby0+cHRfYmFzZSwKZGlmZiAtLWdpdCBhL2FyY2gv eDg2L3hlbi94ZW4tYXNtLlMgYi9hcmNoL3g4Ni94ZW4veGVuLWFzbS5TCmlu ZGV4IGNhNmVkZmU0YzE0Yi4uYjUxOGYzNmQxY2EyIDEwMDY0NAotLS0gYS9h cmNoL3g4Ni94ZW4veGVuLWFzbS5TCisrKyBiL2FyY2gveDg2L3hlbi94ZW4t YXNtLlMKQEAgLTIwLDkgKzIwLDMyIEBACiAKICNpbmNsdWRlIDxsaW51eC9p bml0Lmg+CiAjaW5jbHVkZSA8bGludXgvbGlua2FnZS5oPgorI2luY2x1ZGUg PGxpbnV4L29ianRvb2wuaD4KICNpbmNsdWRlIDwuLi9lbnRyeS9jYWxsaW5n Lmg+CiAKIC5wdXNoc2VjdGlvbiAubm9pbnN0ci50ZXh0LCAiYXgiCisvKgor ICogUFYgaHlwZXJjYWxsIGludGVyZmFjZSB0byB0aGUgaHlwZXJ2aXNvci4K KyAqCisgKiBDYWxsZWQgdmlhIGlubGluZSBhc20oKSwgc28gYmV0dGVyIHBy ZXNlcnZlICVyY3ggYW5kICVyMTEuCisgKgorICogSW5wdXQ6CisgKgklZWF4 OiBoeXBlcmNhbGwgbnVtYmVyCisgKgklcmRpLCAlcnNpLCAlcmR4LCAlcjEw LCAlcjg6IGFyZ3MgMS4uNSBmb3IgdGhlIGh5cGVyY2FsbAorICogT3V0cHV0 OiAlcmF4CisgKi8KK1NZTV9GVU5DX1NUQVJUKHhlbl9oeXBlcmNhbGxfcHYp CisJQU5OT1RBVEVfTk9FTkRCUgorCXB1c2ggJXJjeAorCXB1c2ggJXIxMQor CVVOV0lORF9ISU5UX1NBVkUKKwlzeXNjYWxsCisJVU5XSU5EX0hJTlRfUkVT VE9SRQorCXBvcCAlcjExCisJcG9wICVyY3gKKwlSRVQKK1NZTV9GVU5DX0VO RCh4ZW5faHlwZXJjYWxsX3B2KQorCiAvKgogICogRGlzYWJsaW5nIGV2ZW50 cyBpcyBzaW1wbHkgYSBtYXR0ZXIgb2YgbWFraW5nIHRoZSBldmVudCBtYXNr CiAgKiBub24temVyby4KZGlmZiAtLWdpdCBhL2FyY2gveDg2L3hlbi94ZW4t aGVhZC5TIGIvYXJjaC94ODYveGVuL3hlbi1oZWFkLlMKaW5kZXggN2Y2YzY5 ZGJiODE2Li5jMTczYmE2NzQwZTkgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2L3hl bi94ZW4taGVhZC5TCisrKyBiL2FyY2gveDg2L3hlbi94ZW4taGVhZC5TCkBA IC02LDkgKzYsMTEgQEAKIAogI2luY2x1ZGUgPGxpbnV4L2VsZm5vdGUuaD4K ICNpbmNsdWRlIDxsaW51eC9pbml0Lmg+CisjaW5jbHVkZSA8bGludXgvaW5z dHJ1bWVudGF0aW9uLmg+CiAKICNpbmNsdWRlIDxhc20vYm9vdC5oPgogI2lu Y2x1ZGUgPGFzbS9hc20uaD4KKyNpbmNsdWRlIDxhc20vZnJhbWUuaD4KICNp bmNsdWRlIDxhc20vbXNyLmg+CiAjaW5jbHVkZSA8YXNtL3BhZ2VfdHlwZXMu aD4KICNpbmNsdWRlIDxhc20vcGVyY3B1Lmg+CkBAIC04Nyw2ICs4OSw4NyBA QCBTWU1fQ09ERV9FTkQoeGVuX2NwdV9icmluZ3VwX2FnYWluKQogI2VuZGlm CiAjZW5kaWYKIAorCS5wdXNoc2VjdGlvbiAubm9pbnN0ci50ZXh0LCAiYXgi CisvKgorICogWGVuIGh5cGVyY2FsbCBpbnRlcmZhY2UgdG8gdGhlIGh5cGVy dmlzb3IuCisgKgorICogSW5wdXQ6CisgKiAgICAgJWVheDogaHlwZXJjYWxs IG51bWJlcgorICogICAzMi1iaXQ6CisgKiAgICAgJWVieCwgJWVjeCwgJWVk eCwgJWVzaSwgJWVkaTogYXJncyAxLi41IGZvciB0aGUgaHlwZXJjYWxsCisg KiAgIDY0LWJpdDoKKyAqICAgICAlcmRpLCAlcnNpLCAlcmR4LCAlcjEwLCAl cjg6IGFyZ3MgMS4uNSBmb3IgdGhlIGh5cGVyY2FsbAorICogT3V0cHV0OiAl W2VyXWF4CisgKi8KK1NZTV9GVU5DX1NUQVJUKHhlbl9oeXBlcmNhbGxfaHZt KQorCUVOREJSCisJRlJBTUVfQkVHSU4KKwkvKiBTYXZlIGFsbCByZWxldmFu dCByZWdpc3RlcnMgKGNhbGxlciBzYXZlIGFuZCBhcmd1bWVudHMpLiAqLwor I2lmZGVmIENPTkZJR19YODZfMzIKKwlwdXNoICVlYXgKKwlwdXNoICVlYngK KwlwdXNoICVlY3gKKwlwdXNoICVlZHgKKwlwdXNoICVlc2kKKwlwdXNoICVl ZGkKKyNlbHNlCisJcHVzaCAlcmF4CisJcHVzaCAlcmN4CisJcHVzaCAlcmR4 CisJcHVzaCAlcmRpCisJcHVzaCAlcnNpCisJcHVzaCAlcjExCisJcHVzaCAl cjEwCisJcHVzaCAlcjkKKwlwdXNoICVyOAorI2lmZGVmIENPTkZJR19GUkFN RV9QT0lOVEVSCisJcHVzaHEgJDAJLyogRHVtbXkgcHVzaCBmb3Igc3RhY2sg YWxpZ25tZW50LiAqLworI2VuZGlmCisjZW5kaWYKKwkvKiBTZXQgdGhlIHZl bmRvciBzcGVjaWZpYyBmdW5jdGlvbi4gKi8KKwljYWxsIF9feGVuX2h5cGVy Y2FsbF9zZXRmdW5jCisJLyogU2V0IFpGID0gMSBpZiBBTUQsIFJlc3RvcmUg c2F2ZWQgcmVnaXN0ZXJzLiAqLworI2lmZGVmIENPTkZJR19YODZfMzIKKwls ZWEgeGVuX2h5cGVyY2FsbF9hbWQsICVlYngKKwljbXAgJWVheCwgJWVieAor CXBvcCAlZWRpCisJcG9wICVlc2kKKwlwb3AgJWVkeAorCXBvcCAlZWN4CisJ cG9wICVlYngKKwlwb3AgJWVheAorI2Vsc2UKKwlsZWEgeGVuX2h5cGVyY2Fs bF9hbWQoJXJpcCksICVyYngKKwljbXAgJXJheCwgJXJieAorI2lmZGVmIENP TkZJR19GUkFNRV9QT0lOVEVSCisJcG9wICVyYXgJLyogRHVtbXkgcG9wLiAq LworI2VuZGlmCisJcG9wICVyOAorCXBvcCAlcjkKKwlwb3AgJXIxMAorCXBv cCAlcjExCisJcG9wICVyc2kKKwlwb3AgJXJkaQorCXBvcCAlcmR4CisJcG9w ICVyY3gKKwlwb3AgJXJheAorI2VuZGlmCisJLyogVXNlIGNvcnJlY3QgaHlw ZXJjYWxsIGZ1bmN0aW9uLiAqLworCWp6IHhlbl9oeXBlcmNhbGxfYW1kCisJ am1wIHhlbl9oeXBlcmNhbGxfaW50ZWwKK1NZTV9GVU5DX0VORCh4ZW5faHlw ZXJjYWxsX2h2bSkKKworU1lNX0ZVTkNfU1RBUlQoeGVuX2h5cGVyY2FsbF9h bWQpCisJdm1tY2FsbAorCVJFVAorU1lNX0ZVTkNfRU5EKHhlbl9oeXBlcmNh bGxfYW1kKQorCitTWU1fRlVOQ19TVEFSVCh4ZW5faHlwZXJjYWxsX2ludGVs KQorCXZtY2FsbAorCVJFVAorU1lNX0ZVTkNfRU5EKHhlbl9oeXBlcmNhbGxf aW50ZWwpCisJLnBvcHNlY3Rpb24KKwogCUVMRk5PVEUoWGVuLCBYRU5fRUxG Tk9URV9HVUVTVF9PUywgICAgICAgLmFzY2l6ICJsaW51eCIpCiAJRUxGTk9U RShYZW4sIFhFTl9FTEZOT1RFX0dVRVNUX1ZFUlNJT04sICAuYXNjaXogIjIu NiIpCiAJRUxGTk9URShYZW4sIFhFTl9FTEZOT1RFX1hFTl9WRVJTSU9OLCAg ICAuYXNjaXogInhlbi0zLjAiKQpkaWZmIC0tZ2l0IGEvYXJjaC94ODYveGVu L3hlbi1vcHMuaCBiL2FyY2gveDg2L3hlbi94ZW4tb3BzLmgKaW5kZXggZTFi NzgyZTgyM2U2Li42M2MxM2EyY2NmNTUgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2 L3hlbi94ZW4tb3BzLmgKKysrIGIvYXJjaC94ODYveGVuL3hlbi1vcHMuaApA QCAtMzI2LDQgKzMyNiwxMyBAQCBzdGF0aWMgaW5saW5lIHZvaWQgeGVuX3Nt cF9pbnRyX2ZyZWVfcHYodW5zaWduZWQgaW50IGNwdSkge30KIHN0YXRpYyBp bmxpbmUgdm9pZCB4ZW5fc21wX2NvdW50X2NwdXModm9pZCkgeyB9CiAjZW5k aWYgLyogQ09ORklHX1NNUCAqLwogCisjaWZkZWYgQ09ORklHX1hFTl9QVgor dm9pZCB4ZW5faHlwZXJjYWxsX3B2KHZvaWQpOworI2VuZGlmCit2b2lkIHhl bl9oeXBlcmNhbGxfaHZtKHZvaWQpOwordm9pZCB4ZW5faHlwZXJjYWxsX2Ft ZCh2b2lkKTsKK3ZvaWQgeGVuX2h5cGVyY2FsbF9pbnRlbCh2b2lkKTsKK3Zv aWQgeGVuX2h5cGVyY2FsbF9zZXRmdW5jKHZvaWQpOwordm9pZCAqX194ZW5f aHlwZXJjYWxsX3NldGZ1bmModm9pZCk7CisKICNlbmRpZiAvKiBYRU5fT1BT X0ggKi8KLS0gCjIuNDMuMAoK --=separator Content-Type: application/octet-stream; name="xsa466-linux-06.patch" Content-Disposition: attachment; filename="xsa466-linux-06.patch" Content-Transfer-Encoding: base64 RnJvbSBiMWMyY2I4NmY0YTc4NjE0ODBhZDU0YmI5YTU4ZGYzY2JlYmY4ZTky IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IFRodSwgMTcgT2N0IDIwMjQgMTQ6 NDc6MTMgKzAyMDAKU3ViamVjdDogeDg2L3hlbjogdXNlIG5ldyBoeXBlcmNh bGwgZnVuY3Rpb25zIGluc3RlYWQgb2YgaHlwZXJjYWxsCiBwYWdlCgpDYWxs IHRoZSBYZW4gaHlwZXJ2aXNvciB2aWEgdGhlIG5ldyB4ZW5faHlwZXJjYWxs X2Z1bmMgc3RhdGljLWNhbGwKaW5zdGVhZCBvZiB0aGUgaHlwZXJjYWxsIHBh Z2UuCgpUaGlzIGlzIHBhcnQgb2YgWFNBLTQ2NiAvIENWRS0yMDI0LTUzMjQx LgoKUmVwb3J0ZWQtYnk6IEFuZHJldyBDb29wZXIgPGFuZHJldy5jb29wZXIz QGNpdHJpeC5jb20+ClNpZ25lZC1vZmYtYnk6IEp1ZXJnZW4gR3Jvc3MgPGpn cm9zc0BzdXNlLmNvbT4KQ28tZGV2ZWxvcGVkLWJ5OiBQZXRlciBaaWpsc3Ry YSA8cGV0ZXJ6QGluZnJhZGVhZC5vcmc+CkNvLWRldmVsb3BlZC1ieTogSm9z aCBQb2ltYm9ldWYgPGpwb2ltYm9lQHJlZGhhdC5jb20+Ci0tLQogYXJjaC94 ODYvaW5jbHVkZS9hc20veGVuL2h5cGVyY2FsbC5oIHwgMzMgKysrKysrKysr KysrKysrKystLS0tLS0tLS0tLQogMSBmaWxlIGNoYW5nZWQsIDIwIGluc2Vy dGlvbnMoKyksIDEzIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2FyY2gv eDg2L2luY2x1ZGUvYXNtL3hlbi9oeXBlcmNhbGwuaCBiL2FyY2gveDg2L2lu Y2x1ZGUvYXNtL3hlbi9oeXBlcmNhbGwuaAppbmRleCA2YjRkZDRkZTA4YTYu LjdkNWY4YWQ2Njc3NCAxMDA2NDQKLS0tIGEvYXJjaC94ODYvaW5jbHVkZS9h c20veGVuL2h5cGVyY2FsbC5oCisrKyBiL2FyY2gveDg2L2luY2x1ZGUvYXNt L3hlbi9oeXBlcmNhbGwuaApAQCAtMzksOSArMzksMTEgQEAKICNpbmNsdWRl IDxsaW51eC9zdHJpbmcuaD4KICNpbmNsdWRlIDxsaW51eC90eXBlcy5oPgog I2luY2x1ZGUgPGxpbnV4L3BndGFibGUuaD4KKyNpbmNsdWRlIDxsaW51eC9p bnN0cnVtZW50YXRpb24uaD4KIAogI2luY2x1ZGUgPHRyYWNlL2V2ZW50cy94 ZW4uaD4KIAorI2luY2x1ZGUgPGFzbS9hbHRlcm5hdGl2ZS5oPgogI2luY2x1 ZGUgPGFzbS9wYWdlLmg+CiAjaW5jbHVkZSA8YXNtL3NtYXAuaD4KICNpbmNs dWRlIDxhc20vbm9zcGVjLWJyYW5jaC5oPgpAQCAtOTEsOSArOTMsMTcgQEAg ZXh0ZXJuIHN0cnVjdCB7IGNoYXIgX2VudHJ5WzMyXTsgfSBoeXBlcmNhbGxf cGFnZVtdOwogdm9pZCB4ZW5faHlwZXJjYWxsX2Z1bmModm9pZCk7CiBERUNM QVJFX1NUQVRJQ19DQUxMKHhlbl9oeXBlcmNhbGwsIHhlbl9oeXBlcmNhbGxf ZnVuYyk7CiAKLSNkZWZpbmUgX19IWVBFUkNBTEwJCSJjYWxsIGh5cGVyY2Fs bF9wYWdlKyVjW29mZnNldF0iCi0jZGVmaW5lIF9fSFlQRVJDQUxMX0VOVFJZ KHgpCQkJCQkJXAotCVtvZmZzZXRdICJpIiAoX19IWVBFUlZJU09SXyMjeCAq IHNpemVvZihoeXBlcmNhbGxfcGFnZVswXSkpCisjaWZkZWYgTU9EVUxFCisj ZGVmaW5lIF9fQUREUkVTU0FCTEVfeGVuX2h5cGVyY2FsbAorI2Vsc2UKKyNk ZWZpbmUgX19BRERSRVNTQUJMRV94ZW5faHlwZXJjYWxsIF9fQUREUkVTU0FC TEVfQVNNX1NUUihfX1NDS19feGVuX2h5cGVyY2FsbCkKKyNlbmRpZgorCisj ZGVmaW5lIF9fSFlQRVJDQUxMCQkJCQlcCisJX19BRERSRVNTQUJMRV94ZW5f aHlwZXJjYWxsCQkJXAorCSJjYWxsIF9fU0NUX194ZW5faHlwZXJjYWxsIgor CisjZGVmaW5lIF9fSFlQRVJDQUxMX0VOVFJZKHgpCSJhIiAoeCkKIAogI2lm ZGVmIENPTkZJR19YODZfMzIKICNkZWZpbmUgX19IWVBFUkNBTExfUkVUUkVH CSJlYXgiCkBAIC0xNTEsNyArMTYxLDcgQEAgREVDTEFSRV9TVEFUSUNfQ0FM TCh4ZW5faHlwZXJjYWxsLCB4ZW5faHlwZXJjYWxsX2Z1bmMpOwogCV9fSFlQ RVJDQUxMXzBBUkcoKTsJCQkJCQlcCiAJYXNtIHZvbGF0aWxlIChfX0hZUEVS Q0FMTAkJCQkJXAogCQkgICAgICA6IF9fSFlQRVJDQUxMXzBQQVJBTQkJCQlc Ci0JCSAgICAgIDogX19IWVBFUkNBTExfRU5UUlkobmFtZSkJCQkJXAorCQkg ICAgICA6IF9fSFlQRVJDQUxMX0VOVFJZKF9fSFlQRVJWSVNPUl8gIyMgbmFt ZSkJXAogCQkgICAgICA6IF9fSFlQRVJDQUxMX0NMT0JCRVIwKTsJCQkJXAog CSh0eXBlKV9fcmVzOwkJCQkJCQlcCiB9KQpAQCAtMTYyLDcgKzE3Miw3IEBA IERFQ0xBUkVfU1RBVElDX0NBTEwoeGVuX2h5cGVyY2FsbCwgeGVuX2h5cGVy Y2FsbF9mdW5jKTsKIAlfX0hZUEVSQ0FMTF8xQVJHKGExKTsJCQkJCQlcCiAJ YXNtIHZvbGF0aWxlIChfX0hZUEVSQ0FMTAkJCQkJXAogCQkgICAgICA6IF9f SFlQRVJDQUxMXzFQQVJBTQkJCQlcCi0JCSAgICAgIDogX19IWVBFUkNBTExf RU5UUlkobmFtZSkJCQkJXAorCQkgICAgICA6IF9fSFlQRVJDQUxMX0VOVFJZ KF9fSFlQRVJWSVNPUl8gIyMgbmFtZSkJXAogCQkgICAgICA6IF9fSFlQRVJD QUxMX0NMT0JCRVIxKTsJCQkJXAogCSh0eXBlKV9fcmVzOwkJCQkJCQlcCiB9 KQpAQCAtMTczLDcgKzE4Myw3IEBAIERFQ0xBUkVfU1RBVElDX0NBTEwoeGVu X2h5cGVyY2FsbCwgeGVuX2h5cGVyY2FsbF9mdW5jKTsKIAlfX0hZUEVSQ0FM TF8yQVJHKGExLCBhMik7CQkJCQlcCiAJYXNtIHZvbGF0aWxlIChfX0hZUEVS Q0FMTAkJCQkJXAogCQkgICAgICA6IF9fSFlQRVJDQUxMXzJQQVJBTQkJCQlc Ci0JCSAgICAgIDogX19IWVBFUkNBTExfRU5UUlkobmFtZSkJCQkJXAorCQkg ICAgICA6IF9fSFlQRVJDQUxMX0VOVFJZKF9fSFlQRVJWSVNPUl8gIyMgbmFt ZSkJXAogCQkgICAgICA6IF9fSFlQRVJDQUxMX0NMT0JCRVIyKTsJCQkJXAog CSh0eXBlKV9fcmVzOwkJCQkJCQlcCiB9KQpAQCAtMTg0LDcgKzE5NCw3IEBA IERFQ0xBUkVfU1RBVElDX0NBTEwoeGVuX2h5cGVyY2FsbCwgeGVuX2h5cGVy Y2FsbF9mdW5jKTsKIAlfX0hZUEVSQ0FMTF8zQVJHKGExLCBhMiwgYTMpOwkJ CQkJXAogCWFzbSB2b2xhdGlsZSAoX19IWVBFUkNBTEwJCQkJCVwKIAkJICAg ICAgOiBfX0hZUEVSQ0FMTF8zUEFSQU0JCQkJXAotCQkgICAgICA6IF9fSFlQ RVJDQUxMX0VOVFJZKG5hbWUpCQkJCVwKKwkJICAgICAgOiBfX0hZUEVSQ0FM TF9FTlRSWShfX0hZUEVSVklTT1JfICMjIG5hbWUpCVwKIAkJICAgICAgOiBf X0hZUEVSQ0FMTF9DTE9CQkVSMyk7CQkJCVwKIAkodHlwZSlfX3JlczsJCQkJ CQkJXAogfSkKQEAgLTE5NSw3ICsyMDUsNyBAQCBERUNMQVJFX1NUQVRJQ19D QUxMKHhlbl9oeXBlcmNhbGwsIHhlbl9oeXBlcmNhbGxfZnVuYyk7CiAJX19I WVBFUkNBTExfNEFSRyhhMSwgYTIsIGEzLCBhNCk7CQkJCVwKIAlhc20gdm9s YXRpbGUgKF9fSFlQRVJDQUxMCQkJCQlcCiAJCSAgICAgIDogX19IWVBFUkNB TExfNFBBUkFNCQkJCVwKLQkJICAgICAgOiBfX0hZUEVSQ0FMTF9FTlRSWShu YW1lKQkJCQlcCisJCSAgICAgIDogX19IWVBFUkNBTExfRU5UUlkoX19IWVBF UlZJU09SXyAjIyBuYW1lKQlcCiAJCSAgICAgIDogX19IWVBFUkNBTExfQ0xP QkJFUjQpOwkJCQlcCiAJKHR5cGUpX19yZXM7CQkJCQkJCVwKIH0pCkBAIC0y MDksMTIgKzIxOSw5IEBAIHhlbl9zaW5nbGVfY2FsbCh1bnNpZ25lZCBpbnQg Y2FsbCwKIAlfX0hZUEVSQ0FMTF9ERUNMUzsKIAlfX0hZUEVSQ0FMTF81QVJH KGExLCBhMiwgYTMsIGE0LCBhNSk7CiAKLQlpZiAoY2FsbCA+PSBQQUdFX1NJ WkUgLyBzaXplb2YoaHlwZXJjYWxsX3BhZ2VbMF0pKQotCQlyZXR1cm4gLUVJ TlZBTDsKLQotCWFzbSB2b2xhdGlsZShDQUxMX05PU1BFQworCWFzbSB2b2xh dGlsZShfX0hZUEVSQ0FMTAogCQkgICAgIDogX19IWVBFUkNBTExfNVBBUkFN Ci0JCSAgICAgOiBbdGh1bmtfdGFyZ2V0XSAiYSIgKCZoeXBlcmNhbGxfcGFn ZVtjYWxsXSkKKwkJICAgICA6IF9fSFlQRVJDQUxMX0VOVFJZKGNhbGwpCiAJ CSAgICAgOiBfX0hZUEVSQ0FMTF9DTE9CQkVSNSk7CiAKIAlyZXR1cm4gKGxv bmcpX19yZXM7Ci0tIAoyLjQzLjAKCg== --=separator Content-Type: application/octet-stream; name="xsa466-linux-07.patch" Content-Disposition: attachment; filename="xsa466-linux-07.patch" Content-Transfer-Encoding: base64 RnJvbSA3ZmEwZGE1MzczNjg1ZTdlZDI0OWFmM2ZhMzE3YWIxZTFiYThiMGE2 IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBKdWVyZ2VuIEdyb3Nz IDxqZ3Jvc3NAc3VzZS5jb20+CkRhdGU6IFRodSwgMTcgT2N0IDIwMjQgMTU6 Mjc6MzEgKzAyMDAKU3ViamVjdDogeDg2L3hlbjogcmVtb3ZlIGh5cGVyY2Fs bCBwYWdlCgpUaGUgaHlwZXJjYWxsIHBhZ2UgaXMgbm8gbG9uZ2VyIG5lZWRl ZC4gSXQgY2FuIGJlIHJlbW92ZWQsIGFzIGZyb20gdGhlClhlbiBwZXJzcGVj dGl2ZSBpdCBpcyBvcHRpb25hbC4KCkJ1dCwgZnJvbSBMaW51eCdzIHBlcnNw ZWN0aXZlLCBpdCByZW1vdmVzIG5ha2VkIFJFVCBpbnN0cnVjdGlvbnMgdGhh dAplc2NhcGUgdGhlIHNwZWN1bGF0aXZlIHByb3RlY3Rpb25zIHRoYXQgQ2Fs bCBEZXB0aCBUcmFja2luZyBhbmQvb3IKVW50cmFpbiBSZXQgYXJlIHRyeWlu ZyB0byBhY2hpZXZlLgoKVGhpcyBpcyBwYXJ0IG9mIFhTQS00NjYgLyBDVkUt MjAyNC01MzI0MS4KClJlcG9ydGVkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy ZXcuY29vcGVyM0BjaXRyaXguY29tPgpTaWduZWQtb2ZmLWJ5OiBKdWVyZ2Vu IEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+ClJldmlld2VkLWJ5OiBBbmRyZXcg Q29vcGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPgpSZXZpZXdlZC1i eTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgotLS0KIGFyY2gv eDg2L2luY2x1ZGUvYXNtL3hlbi9oeXBlcmNhbGwuaCB8ICAyIC0tCiBhcmNo L3g4Ni9rZXJuZWwvY2FsbHRodW5rcy5jICAgICAgICAgfCAgNSAtLS0tLQog YXJjaC94ODYva2VybmVsL3ZtbGludXgubGRzLlMgICAgICAgIHwgIDQgLS0t LQogYXJjaC94ODYveGVuL2VubGlnaHRlbi5jICAgICAgICAgICAgIHwgIDIg LS0KIGFyY2gveDg2L3hlbi9lbmxpZ2h0ZW5faHZtLmMgICAgICAgICB8ICA5 ICstLS0tLS0tLQogYXJjaC94ODYveGVuL2VubGlnaHRlbl9wdmguYyAgICAg ICAgIHwgIDcgLS0tLS0tLQogYXJjaC94ODYveGVuL3hlbi1oZWFkLlMgICAg ICAgICAgICAgIHwgMjQgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiA3IGZp bGVzIGNoYW5nZWQsIDEgaW5zZXJ0aW9uKCspLCA1MiBkZWxldGlvbnMoLSkK CmRpZmYgLS1naXQgYS9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vaHlwZXJj YWxsLmggYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vaHlwZXJjYWxsLmgK aW5kZXggN2Q1ZjhhZDY2Nzc0Li45Nzc3MWI5ZDMzYWYgMTAwNjQ0Ci0tLSBh L2FyY2gveDg2L2luY2x1ZGUvYXNtL3hlbi9oeXBlcmNhbGwuaAorKysgYi9h cmNoL3g4Ni9pbmNsdWRlL2FzbS94ZW4vaHlwZXJjYWxsLmgKQEAgLTg4LDgg Kzg4LDYgQEAgc3RydWN0IHhlbl9kbV9vcF9idWY7CiAgKiB0aGVyZSBhcmVu J3QgbW9yZSB0aGFuIDUgYXJndW1lbnRzLi4uKQogICovCiAKLWV4dGVybiBz dHJ1Y3QgeyBjaGFyIF9lbnRyeVszMl07IH0gaHlwZXJjYWxsX3BhZ2VbXTsK LQogdm9pZCB4ZW5faHlwZXJjYWxsX2Z1bmModm9pZCk7CiBERUNMQVJFX1NU QVRJQ19DQUxMKHhlbl9oeXBlcmNhbGwsIHhlbl9oeXBlcmNhbGxfZnVuYyk7 CiAKZGlmZiAtLWdpdCBhL2FyY2gveDg2L2tlcm5lbC9jYWxsdGh1bmtzLmMg Yi9hcmNoL3g4Ni9rZXJuZWwvY2FsbHRodW5rcy5jCmluZGV4IDQ2NTY0NzQ1 Njc1My4uZjE3ZDE2NjA3ODgyIDEwMDY0NAotLS0gYS9hcmNoL3g4Ni9rZXJu ZWwvY2FsbHRodW5rcy5jCisrKyBiL2FyY2gveDg2L2tlcm5lbC9jYWxsdGh1 bmtzLmMKQEAgLTE0MiwxMSArMTQyLDYgQEAgc3RhdGljIGJvb2wgc2tpcF9h ZGRyKHZvaWQgKmRlc3QpCiAJaWYgKGRlc3QgPj0gKHZvaWQgKilyZWxvY2F0 ZV9rZXJuZWwgJiYKIAkgICAgZGVzdCA8ICh2b2lkKilyZWxvY2F0ZV9rZXJu ZWwgKyBLRVhFQ19DT05UUk9MX0NPREVfTUFYX1NJWkUpCiAJCXJldHVybiB0 cnVlOwotI2VuZGlmCi0jaWZkZWYgQ09ORklHX1hFTgotCWlmIChkZXN0ID49 ICh2b2lkICopaHlwZXJjYWxsX3BhZ2UgJiYKLQkgICAgZGVzdCA8ICh2b2lk KiloeXBlcmNhbGxfcGFnZSArIFBBR0VfU0laRSkKLQkJcmV0dXJuIHRydWU7 CiAjZW5kaWYKIAlyZXR1cm4gZmFsc2U7CiB9CmRpZmYgLS1naXQgYS9hcmNo L3g4Ni9rZXJuZWwvdm1saW51eC5sZHMuUyBiL2FyY2gveDg2L2tlcm5lbC92 bWxpbnV4Lmxkcy5TCmluZGV4IGZhYjNhYzlhNDU3NC4uNmExNzM5NmM4MTc0 IDEwMDY0NAotLS0gYS9hcmNoL3g4Ni9rZXJuZWwvdm1saW51eC5sZHMuUwor KysgYi9hcmNoL3g4Ni9rZXJuZWwvdm1saW51eC5sZHMuUwpAQCAtNTE5LDE0 ICs1MTksMTAgQEAgSU5JVF9QRVJfQ1BVKGlycV9zdGFja19iYWNraW5nX3N0 b3JlKTsKICAqIGxpbmtlciB3aWxsIG5ldmVyIG1hcmsgYXMgcmVsb2NhdGFi bGUuIChVc2luZyBqdXN0IEFCU09MVVRFKCkgaXMgbm90CiAgKiBzdWZmaWNp ZW50IGZvciB0aGF0KS4KICAqLwotI2lmZGVmIENPTkZJR19YRU4KICNpZmRl ZiBDT05GSUdfWEVOX1BWCiB4ZW5fZWxmbm90ZV9lbnRyeV92YWx1ZSA9CiAJ QUJTT0xVVEUoeGVuX2VsZm5vdGVfZW50cnkpICsgQUJTT0xVVEUoc3RhcnR1 cF94ZW4pOwogI2VuZGlmCi14ZW5fZWxmbm90ZV9oeXBlcmNhbGxfcGFnZV92 YWx1ZSA9Ci0JQUJTT0xVVEUoeGVuX2VsZm5vdGVfaHlwZXJjYWxsX3BhZ2Up ICsgQUJTT0xVVEUoaHlwZXJjYWxsX3BhZ2UpOwotI2VuZGlmCiAjaWZkZWYg Q09ORklHX1BWSAogeGVuX2VsZm5vdGVfcGh5czMyX2VudHJ5X3ZhbHVlID0K IAlBQlNPTFVURSh4ZW5fZWxmbm90ZV9waHlzMzJfZW50cnkpICsgQUJTT0xV VEUocHZoX3N0YXJ0X3hlbiAtIExPQURfT0ZGU0VUKTsKZGlmZiAtLWdpdCBh L2FyY2gveDg2L3hlbi9lbmxpZ2h0ZW4uYyBiL2FyY2gveDg2L3hlbi9lbmxp Z2h0ZW4uYwppbmRleCAxODg3NDM1YWYyZmIuLjQzZGNkOGM3YmFkYyAxMDA2 NDQKLS0tIGEvYXJjaC94ODYveGVuL2VubGlnaHRlbi5jCisrKyBiL2FyY2gv eDg2L3hlbi9lbmxpZ2h0ZW4uYwpAQCAtMjIsOCArMjIsNiBAQAogCiAjaW5j bHVkZSAieGVuLW9wcy5oIgogCi1FWFBPUlRfU1lNQk9MX0dQTChoeXBlcmNh bGxfcGFnZSk7Ci0KIERFRklORV9TVEFUSUNfQ0FMTCh4ZW5faHlwZXJjYWxs LCB4ZW5faHlwZXJjYWxsX2h2bSk7CiBFWFBPUlRfU1RBVElDX0NBTExfVFJB TVAoeGVuX2h5cGVyY2FsbCk7CiAKZGlmZiAtLWdpdCBhL2FyY2gveDg2L3hl bi9lbmxpZ2h0ZW5faHZtLmMgYi9hcmNoL3g4Ni94ZW4vZW5saWdodGVuX2h2 bS5jCmluZGV4IDk3M2E3NGZjOTY2YS4uZmU1N2ZmODVkMDA0IDEwMDY0NAot LS0gYS9hcmNoL3g4Ni94ZW4vZW5saWdodGVuX2h2bS5jCisrKyBiL2FyY2gv eDg2L3hlbi9lbmxpZ2h0ZW5faHZtLmMKQEAgLTEwNiwxNSArMTA2LDggQEAg c3RhdGljIHZvaWQgX19pbml0IGluaXRfaHZtX3B2X2luZm8odm9pZCkKIAkv KiBQVkggc2V0IHVwIGh5cGVyY2FsbCBwYWdlIGluIHhlbl9wcmVwYXJlX3B2 aCgpLiAqLwogCWlmICh4ZW5fcHZoX2RvbWFpbigpKQogCQlwdl9pbmZvLm5h bWUgPSAiWGVuIFBWSCI7Ci0JZWxzZSB7Ci0JCXU2NCBwZm47Ci0JCXVpbnQz Ml90IG1zcjsKLQorCWVsc2UKIAkJcHZfaW5mby5uYW1lID0gIlhlbiBIVk0i OwotCQltc3IgPSBjcHVpZF9lYngoYmFzZSArIDIpOwotCQlwZm4gPSBfX3Bh KGh5cGVyY2FsbF9wYWdlKTsKLQkJd3Jtc3Jfc2FmZShtc3IsICh1MzIpcGZu LCAodTMyKShwZm4gPj4gMzIpKTsKLQl9CiAKIAl4ZW5fc2V0dXBfZmVhdHVy ZXMoKTsKIApkaWZmIC0tZ2l0IGEvYXJjaC94ODYveGVuL2VubGlnaHRlbl9w dmguYyBiL2FyY2gveDg2L3hlbi9lbmxpZ2h0ZW5fcHZoLmMKaW5kZXggYmY2 OGMzMjlmYzAxLi4wZTNkOTMwYmNiODkgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2 L3hlbi9lbmxpZ2h0ZW5fcHZoLmMKKysrIGIvYXJjaC94ODYveGVuL2VubGln aHRlbl9wdmguYwpAQCAtMTI5LDE3ICsxMjksMTAgQEAgc3RhdGljIHZvaWQg X19pbml0IHB2aF9hcmNoX3NldHVwKHZvaWQpCiAKIHZvaWQgX19pbml0IHhl bl9wdmhfaW5pdChzdHJ1Y3QgYm9vdF9wYXJhbXMgKmJvb3RfcGFyYW1zKQog ewotCXUzMiBtc3I7Ci0JdTY0IHBmbjsKLQogCXhlbl9wdmggPSAxOwogCXhl bl9kb21haW5fdHlwZSA9IFhFTl9IVk1fRE9NQUlOOwogCXhlbl9zdGFydF9m bGFncyA9IHB2aF9zdGFydF9pbmZvLmZsYWdzOwogCi0JbXNyID0gY3B1aWRf ZWJ4KHhlbl9jcHVpZF9iYXNlKCkgKyAyKTsKLQlwZm4gPSBfX3BhKGh5cGVy Y2FsbF9wYWdlKTsKLQl3cm1zcl9zYWZlKG1zciwgKHUzMilwZm4sICh1MzIp KHBmbiA+PiAzMikpOwotCiAJeDg2X2luaXQub2VtLmFyY2hfc2V0dXAgPSBw dmhfYXJjaF9zZXR1cDsKIAl4ODZfaW5pdC5vZW0uYmFubmVyID0geGVuX2Jh bm5lcjsKIApkaWZmIC0tZ2l0IGEvYXJjaC94ODYveGVuL3hlbi1oZWFkLlMg Yi9hcmNoL3g4Ni94ZW4veGVuLWhlYWQuUwppbmRleCBjMTczYmE2NzQwZTku LjkyNTI2NTJhZmU1OSAxMDA2NDQKLS0tIGEvYXJjaC94ODYveGVuL3hlbi1o ZWFkLlMKKysrIGIvYXJjaC94ODYveGVuL3hlbi1oZWFkLlMKQEAgLTIyLDI4 ICsyMiw2IEBACiAjaW5jbHVkZSA8eGVuL2ludGVyZmFjZS94ZW4tbWNhLmg+ CiAjaW5jbHVkZSA8YXNtL3hlbi9pbnRlcmZhY2UuaD4KIAotLnB1c2hzZWN0 aW9uIC5ub2luc3RyLnRleHQsICJheCIKLQkuYmFsaWduIFBBR0VfU0laRQot U1lNX0NPREVfU1RBUlQoaHlwZXJjYWxsX3BhZ2UpCi0JLnJlcHQgKFBBR0Vf U0laRSAvIDMyKQotCQlVTldJTkRfSElOVF9GVU5DCi0JCUFOTk9UQVRFX05P RU5EQlIKLQkJQU5OT1RBVEVfVU5SRVRfU0FGRQotCQlyZXQKLQkJLyoKLQkJ ICogWGVuIHdpbGwgd3JpdGUgdGhlIGh5cGVyY2FsbCBwYWdlLCBhbmQgc29y dCBvdXQgRU5EQlIuCi0JCSAqLwotCQkuc2tpcCAzMSwgMHhjYwotCS5lbmRy Ci0KLSNkZWZpbmUgSFlQRVJDQUxMKG4pIFwKLQkuZXF1IHhlbl9oeXBlcmNh bGxfIyNuLCBoeXBlcmNhbGxfcGFnZSArIF9fSFlQRVJWSVNPUl8jI24gKiAz MjsgXAotCS50eXBlIHhlbl9oeXBlcmNhbGxfIyNuLCBAZnVuY3Rpb247IC5z aXplIHhlbl9oeXBlcmNhbGxfIyNuLCAzMgotI2luY2x1ZGUgPGFzbS94ZW4t aHlwZXJjYWxscy5oPgotI3VuZGVmIEhZUEVSQ0FMTAotU1lNX0NPREVfRU5E KGh5cGVyY2FsbF9wYWdlKQotLnBvcHNlY3Rpb24KLQogI2lmZGVmIENPTkZJ R19YRU5fUFYKIAlfX0lOSVQKIFNZTV9DT0RFX1NUQVJUKHN0YXJ0dXBfeGVu KQpAQCAtMTk5LDggKzE3Nyw2IEBAIFNZTV9GVU5DX0VORCh4ZW5faHlwZXJj YWxsX2ludGVsKQogI2Vsc2UKICMgZGVmaW5lIEZFQVRVUkVTX0RPTTAgMAog I2VuZGlmCi0JRUxGTk9URShYZW4sIFhFTl9FTEZOT1RFX0hZUEVSQ0FMTF9Q QUdFLCAuZ2xvYmwgeGVuX2VsZm5vdGVfaHlwZXJjYWxsX3BhZ2U7Ci0JCXhl bl9lbGZub3RlX2h5cGVyY2FsbF9wYWdlOiBfQVNNX1BUUiB4ZW5fZWxmbm90 ZV9oeXBlcmNhbGxfcGFnZV92YWx1ZSAtIC4pCiAJRUxGTk9URShYZW4sIFhF Tl9FTEZOT1RFX1NVUFBPUlRFRF9GRUFUVVJFUywKIAkJLmxvbmcgRkVBVFVS RVNfUFYgfCBGRUFUVVJFU19QVkggfCBGRUFUVVJFU19ET00wKQogCUVMRk5P VEUoWGVuLCBYRU5fRUxGTk9URV9MT0FERVIsICAgICAgICAgLmFzY2l6ICJn ZW5lcmljIikKLS0gCjIuNDMuMAoK --=separator Content-Type: application/octet-stream; name="xsa466-xen.patch" Content-Disposition: attachment; filename="xsa466-xen.patch" Content-Transfer-Encoding: base64 RnJvbTogQW5kcmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNv bT4KU3ViamVjdDogZG9jcy9ndWVzdC1ndWlkZTogRGlzY3VzcyB3aGVuIG5v dCB1c2UgYSBoeXBlcmNhbGwgcGFnZQoKVGhlIExpbnV4IHJldGh1bmsgYW5k IHNhZmUtcmV0IHNwZWN1bGF0aXZlIHNhZmV0eSB0ZWNobmlxdWVzIGludm9s dmUKdHJhbnNmb3JtaW5nIGByZXRgIHRvIGBqbXAgX194ODZfcmV0dXJuX3Ro dW5rYCBhdCBjb21waWxlIHRpbWUuICBQbGFjaW5nIG5ha2VkCmByZXRgcyBi YWNrIGluIGV4ZWN1dGFibGUgLnRleHQgYnJlYWtzIHRoZXNlIG1pdGlnYXRp b25zLgoKQ0VULUlCVCByZXF1aXJlcyBFTkRCUiBpbnN0cnVjdGlvbnMsIGFu ZCB3aGlsZSB3ZSBjb3VsZCBpbiBwcmluY2lwbGUgZml4IHRoYXQsCnRoZSBu ZWVkIHRvIHNlbGVjdCBiZXR3ZWVuIEVOREJSMzIgb3IgRU5EQlI2NCBtZWFu cyB0aGF0IHRoZSBjb250ZW50cyBvZiB0aGUKaHlwZXJjYWxsIHBhZ2Ugd291 bGQgbmVlZCB0byBiZWNvbWUgbW9yZSBtb2RlLXNwZWNpZmljIHRoYW4gaXQg Y3VycmVudGx5CmlzIChIVk0gaHlwZXJjYWxsIHBhZ2VzIGFyZSBjdXJyZW50 bHkgMzJiaXQgYW5kIDY0Yml0IGNvbXBhdGJpbGUpLiAgSG93ZXZlciwKdGhl cmUncyBubyBmZWFzaWJsZSB3YXkgdG8gbWFrZSBhIGh5cGVyY2FsbCBwYWdl IGNvbXBhdGlibGUgd2l0aCBmaW5lLWdyYWluCkNGSSBzY2hlbWVzIHN1Y2gg YXMgRmluZUlCVC4KCk9TZXMgd2hpY2ggY2FyZSBhYm91dCBlaXRoZXIgb2Yg dGhlc2UgdGhpbmdzIGFyZSBiZXR0ZXIgb2ZmIGF2b2lkaW5nIHRoZQpoeXBl cmNhbGwgcGFnZS4KClRoaXMgaXMgcGFydCBvZiBYU0EtNDY2LgoKU2lnbmVk LW9mZi1ieTogQW5kcmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4 LmNvbT4KUmV2aWV3ZWQtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNl LmNvbT4KCmRpZmYgLS1naXQgYS9kb2NzL2d1ZXN0LWd1aWRlL3g4Ni9oeXBl cmNhbGwtYWJpLnJzdCBiL2RvY3MvZ3Vlc3QtZ3VpZGUveDg2L2h5cGVyY2Fs bC1hYmkucnN0CmluZGV4IDgwMDQxMjJjYTQ5ZC4uNzQ1ZmJiYjY0YTI2IDEw MDY0NAotLS0gYS9kb2NzL2d1ZXN0LWd1aWRlL3g4Ni9oeXBlcmNhbGwtYWJp LnJzdAorKysgYi9kb2NzL2d1ZXN0LWd1aWRlL3g4Ni9oeXBlcmNhbGwtYWJp LnJzdApAQCAtODIsNiArODIsMTMgQEAgVGhlIGh5cGVyY2FsbCBwYWdlIGlz IGEgcGFnZSBvZiBndWVzdCBSQU0gaW50byB3aGljaCBYZW4gd2lsbCB3cml0 ZSBzdWl0YWJsZQogdHJhbnNmZXIgc3R1YnMuICBJdCBpcyBpbnRlbmRlZCBh cyBhIGNvbnZlbmllbmNlIGZvciBndWVzdHMsIGJ1dCB1c2Ugb2YgdGhlCiBo eXBlcmNhbGwgcGFnZSBpcyBub3QgbWFuZGF0b3J5IGZvciBtYWtpbmcgaHlw ZXJjYWxscyB0byBYZW4uCiAKKy4uIG5vdGU6OgorCisgICBUaGVyZSBhcmUg Y2FzZXMgd2hlcmUgYSBoeXBlcmNhbGwgcGFnZSBzaG91bGQgbm90IGJlIHVz ZWQuICBJdCBjb250YWlucworICAgYGByZXRgYCBpbnN0cnVjdGlvbnMgd2hp Y2ggYXJlIG5vdCBjb21wYXRpYmxlIHdpdGggY2VydGFpbiBzcGVjdWxhdGl2 ZQorICAgc2VjdXJpdHkgdGVjaG5pcXVlcywgYW5kIGl0IGRvZXMgbm90IGNv bnRhaW4gYGBlbmRicmBgIGluc3RydWN0aW9ucyB3aGljaAorICAgYXJlIG5l Y2Vzc2FyeSBmb3IgY2VydGFpbiBDb250cm9sLWZsb3cgSW50ZWdyaXR5IHNj aGVtZXMuCisKIENyZWF0aW5nIGEgaHlwZXJjYWxsIHBhZ2UgaXMgYW4gaXNv bGF0ZWQgb3BlcmF0aW9uIGZyb20gWGVuJ3MgcG9pbnQgb2Ygdmlldy4KIEl0 IGlzIHRoZSBndWVzdHMgcmVzcG9uc2liaWxpdHkgdG8gZW5zdXJlIHRoYXQg dGhlIGh5cGVyY2FsbCBwYWdlLCBvbmNlCiB3cml0dGVuIGJ5IFhlbiwgaXMg bWFwcGVkIHdpdGggZXhlY3V0YWJsZSBwZXJtaXNzaW9ucyBzbyBpdCBtYXkg YmUgdXNlZC4K --=separator-- From xen-users-bounces@lists.xenproject.org Tue Dec 24 16:17:15 2024 Return-path: Envelope-to: archives@lists.xen.org Delivery-date: Tue, 24 Dec 2024 16:17:15 +0000 Received: from list by lists.xenproject.org with outflank-mailman.862908.1274561 (Exim 4.92) (envelope-from ) id 1tQ7a4-0004a0-PH; Tue, 24 Dec 2024 16:16:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 862908.1274561; Tue, 24 Dec 2024 16:16:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tQ7a4-0004Zt-Kc; Tue, 24 Dec 2024 16:16:24 +0000 Received: by outflank-mailman (input) for mailman id 862908; Mon, 23 Dec 2024 14:24:24 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tPjM7-0004EQ-VB for xen-users@lists.xen.org; Mon, 23 Dec 2024 14:24:23 +0000 Received: from casper.infradead.org (casper.infradead.org [2001:8b0:10b:1236::1]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 950b4dd2-c139-11ef-99a3-01e77a169b0f; Mon, 23 Dec 2024 15:24:15 +0100 (CET) Received: from 54-240-197-226.amazon.com ([54.240.197.226] helo=freeip.amazon.com) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tPjLw-0000000FeXG-06JL; Mon, 23 Dec 2024 14:24:12 +0000 X-BeenThere: xen-users@lists.xenproject.org List-Id: Xen user discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-users-bounces@lists.xenproject.org Precedence: list Sender: "Xen-users" X-Inumbo-ID: 950b4dd2-c139-11ef-99a3-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:References: In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=2k1YSapLUNBvQU7CZJmozDaJQTyQbwqeYmlBunuUZUQ=; b=mVhynl5x25Tshn1Y3YP2CDgmNy w0WC5QfJE56QUKY8dqpMCBSNn1yail0Y8KmYMONyOq0lFkc33c8+6btsOdY0CoPBCBwVzVlZhyzk+ 7Kq4UGxSXrbfq3u4AtgwuIjd91JKEjxuc1+Gc92VEbNAbEo5oAcr6MExHi1HLWjfercG0ehsA5XUc g1NQ1x9mAwA1WPBTc1srcf//YlDRyuxv1pZkHqUUkirUG9mksxLkZJu+tX2vRo0YVm6OWdBL4V5iM g3cH+Od80iSMq1I7rhNP800kg6OMdxGE8YUzDAvu8egGhtaePU0v9zaYTI32kNGe6A2oe+Cq0Emir 0Sm7LS2A==; Message-ID: <54c892eded2b4ebdda8ee1085c383178f44414ad.camel@infradead.org> Subject: Re: Xen Security Advisory 466 v3 (CVE-2024-53241) - Xen hypercall page unsafe against speculative attacks From: David Woodhouse To: "Xen.org security team" , xen-announce@lists.xen.org, xen-devel@lists.xen.org, xen-users@lists.xen.org, oss-security@lists.openwall.com Cc: "Xen.org security team" Date: Mon, 23 Dec 2024 14:24:10 +0000 In-Reply-To: References: Content-Type: multipart/signed; micalg="sha-256"; protocol="application/pkcs7-signature"; boundary="=-2cuJts2lurvGX3hsXblo" User-Agent: Evolution 3.52.3-0ubuntu1 MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html --=-2cuJts2lurvGX3hsXblo Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2024-12-17 at 12:18 +0000, Xen.org security team wrote: > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Xen Security Advisory CVE-2024-53241 / XSA-466 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 version 3 >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Xen hypercall page unsaf= e against speculative attacks >=20 > UPDATES IN VERSION 3 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Update of patch 5, public release. Can't we even use the hypercall page early in boot? Surely we have to know whether we're running on an Intel or AMD CPU before we get to the point where we can enable any of the new control-flow integrity support? Do we need to jump through those hoops do do that early detection and setup? Enabling the hypercall page is also one of the two points where Xen will 'latch' that the guest is 64-bit, which affects the layout of the shared_info, vcpu_info and runstate structures. The other such latching point is when the guest sets HVM_PARAM_CALLBACK_IRQ, and I *think* that should work in all implementations of the Xen ABI (including QEMU/KVM and EC2). But would want to test. But perhaps it wouldn't hurt for maximal compatibility for Linux to set the hypercall page *anyway*, even if Linux doesn't then use it =E2=80=94 or only uses it during early boot? =20 --=-2cuJts2lurvGX3hsXblo Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCEkQw ggYQMIID+KADAgECAhBNlCwQ1DvglAnFgS06KwZPMA0GCSqGSIb3DQEBDAUAMIGIMQswCQYDVQQG EwJVUzETMBEGA1UECBMKTmV3IEplcnNleTEUMBIGA1UEBxMLSmVyc2V5IENpdHkxHjAcBgNVBAoT FVRoZSBVU0VSVFJVU1QgTmV0d29yazEuMCwGA1UEAxMlVVNFUlRydXN0IFJTQSBDZXJ0aWZpY2F0 aW9uIEF1dGhvcml0eTAeFw0xODExMDIwMDAwMDBaFw0zMDEyMzEyMzU5NTlaMIGWMQswCQYDVQQG EwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYD VQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50 aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC AQEAyjztlApB/975Rrno1jvm2pK/KxBOqhq8gr2+JhwpKirSzZxQgT9tlC7zl6hn1fXjSo5MqXUf ItMltrMaXqcESJuK8dtK56NCSrq4iDKaKq9NxOXFmqXX2zN8HHGjQ2b2Xv0v1L5Nk1MQPKA19xeW QcpGEGFUUd0kN+oHox+L9aV1rjfNiCj3bJk6kJaOPabPi2503nn/ITX5e8WfPnGw4VuZ79Khj1YB rf24k5Ee1sLTHsLtpiK9OjG4iQRBdq6Z/TlVx/hGAez5h36bBJMxqdHLpdwIUkTqT8se3ed0PewD ch/8kHPo5fZl5u1B0ecpq/sDN/5sCG52Ds+QU5O5EwIDAQABo4IBZDCCAWAwHwYDVR0jBBgwFoAU U3m/WqorSs9UgOHYm8Cd8rIDZsswHQYDVR0OBBYEFAnA8vwL2pTbX/4r36iZQs/J4K0AMA4GA1Ud DwEB/wQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDARBgNVHSAECjAIMAYGBFUdIAAwUAYDVR0fBEkwRzBFoEOgQYY/aHR0cDovL2NybC51c2Vy dHJ1c3QuY29tL1VTRVJUcnVzdFJTQUNlcnRpZmljYXRpb25BdXRob3JpdHkuY3JsMHYGCCsGAQUF BwEBBGowaDA/BggrBgEFBQcwAoYzaHR0cDovL2NydC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJT QUFkZFRydXN0Q0EuY3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0G CSqGSIb3DQEBDAUAA4ICAQBBRHUAqznCFfXejpVtMnFojADdF9d6HBA4kMjjsb0XMZHztuOCtKF+ xswhh2GqkW5JQrM8zVlU+A2VP72Ky2nlRA1GwmIPgou74TZ/XTarHG8zdMSgaDrkVYzz1g3nIVO9 IHk96VwsacIvBF8JfqIs+8aWH2PfSUrNxP6Ys7U0sZYx4rXD6+cqFq/ZW5BUfClN/rhk2ddQXyn7 kkmka2RQb9d90nmNHdgKrwfQ49mQ2hWQNDkJJIXwKjYA6VUR/fZUFeCUisdDe/0ABLTI+jheXUV1 eoYV7lNwNBKpeHdNuO6Aacb533JlfeUHxvBz9OfYWUiXu09sMAviM11Q0DuMZ5760CdO2VnpsXP4 KxaYIhvqPqUMWqRdWyn7crItNkZeroXaecG03i3mM7dkiPaCkgocBg0EBYsbZDZ8bsG3a08LwEsL 1Ygz3SBsyECa0waq4hOf/Z85F2w2ZpXfP+w8q4ifwO90SGZZV+HR/Jh6rEaVPDRF/CEGVqR1hiuQ OZ1YL5ezMTX0ZSLwrymUE0pwi/KDaiYB15uswgeIAcA6JzPFf9pLkAFFWs1QNyN++niFhsM47qod x/PL+5jR87myx5uYdBEQkkDc+lKB1Wct6ucXqm2EmsaQ0M95QjTmy+rDWjkDYdw3Ms6mSWE3Bn7i 5ZgtwCLXgAIe5W8mybM2JzCCBhQwggT8oAMCAQICEQDGvhmWZ0DEAx0oURL6O6l+MA0GCSqGSIb3 DQEBCwUAMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYD VQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNlY3RpZ28g UlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTIyMDEwNzAw MDAwMFoXDTI1MDEwNjIzNTk1OVowJDEiMCAGCSqGSIb3DQEJARYTZHdtdzJAaW5mcmFkZWFkLm9y ZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQ3GpC2bomUqk+91wLYBzDMcCj5C9m6 oZaHwvmIdXftOgTbCJXADo6G9T7BBAebw2JV38EINgKpy/ZHh7htyAkWYVoFsFPrwHounto8xTsy SSePMiPlmIdQ10BcVSXMUJ3Juu16GlWOnAMJY2oYfEzmE7uT9YgcBqKCo65pTFmOnR/VVbjJk4K2 xE34GC2nAdUQkPFuyaFisicc6HRMOYXPuF0DuwITEKnjxgNjP+qDrh0db7PAjO1D4d5ftfrsf+kd RR4gKVGSk8Tz2WwvtLAroJM4nXjNPIBJNT4w/FWWc/5qPHJy2U+eITZ5LLE5s45mX2oPFknWqxBo bQZ8a9dsZ3dSPZBvE9ZrmtFLrVrN4eo1jsXgAp1+p7bkfqd3BgBEmfsYWlBXO8rVXfvPgLs32VdV NZxb/CDWPqBsiYv0Hv3HPsz07j5b+/cVoWqyHDKzkaVbxfq/7auNVRmPB3v5SWEsH8xi4Bez2V9U KxfYCnqsjp8RaC2/khxKt0A552Eaxnz/4ly/2C7wkwTQnBmdlFYhAflWKQ03Ufiu8t3iBE3VJbc2 5oMrglj7TRZrmKq3CkbFnX0fyulB+kHimrt6PIWn7kgyl9aelIl6vtbhMA+l0nfrsORMa4kobqQ5 C5rveVgmcIad67EDa+UqEKy/GltUwlSh6xy+TrK1tzDvAgMBAAGjggHMMIIByDAfBgNVHSMEGDAW gBQJwPL8C9qU21/+K9+omULPyeCtADAdBgNVHQ4EFgQUzMeDMcimo0oz8o1R1Nver3ZVpSkwDgYD VR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMC MEAGA1UdIAQ5MDcwNQYMKwYBBAGyMQECAQEBMCUwIwYIKwYBBQUHAgEWF2h0dHBzOi8vc2VjdGln by5jb20vQ1BTMFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9jcmwuc2VjdGlnby5jb20vU2VjdGln b1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1haWxDQS5jcmwwgYoGCCsGAQUFBwEB BH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0aWdvUlNBQ2xpZW50 QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAjBggrBgEFBQcwAYYXaHR0cDovL29j c3Auc2VjdGlnby5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5mcmFkZWFkLm9yZzANBgkqhkiG9w0B AQsFAAOCAQEAyW6MUir5dm495teKqAQjDJwuFCi35h4xgnQvQ/fzPXmtR9t54rpmI2TfyvcKgOXp qa7BGXNFfh1JsqexVkIqZP9uWB2J+uVMD+XZEs/KYNNX2PvIlSPrzIB4Z2wyIGQpaPLlYflrrVFK v9CjT2zdqvy2maK7HKOQRt3BiJbVG5lRiwbbygldcALEV9ChWFfgSXvrWDZspnU3Gjw/rMHrGnql Htlyebp3pf3fSS9kzQ1FVtVIDrL6eqhTwJxe+pXSMMqFiN0whpBtXdyDjzBtQTaZJ7zTT/vlehc/ tDuqZwGHm/YJy883Ll+GP3NvOkgaRGWEuYWJJ6hFCkXYjyR9IzCCBhQwggT8oAMCAQICEQDGvhmW Z0DEAx0oURL6O6l+MA0GCSqGSIb3DQEBCwUAMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3Jl YXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0 ZWQxPjA8BgNVBAMTNVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJl IEVtYWlsIENBMB4XDTIyMDEwNzAwMDAwMFoXDTI1MDEwNjIzNTk1OVowJDEiMCAGCSqGSIb3DQEJ ARYTZHdtdzJAaW5mcmFkZWFkLm9yZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQ3 GpC2bomUqk+91wLYBzDMcCj5C9m6oZaHwvmIdXftOgTbCJXADo6G9T7BBAebw2JV38EINgKpy/ZH h7htyAkWYVoFsFPrwHounto8xTsySSePMiPlmIdQ10BcVSXMUJ3Juu16GlWOnAMJY2oYfEzmE7uT 9YgcBqKCo65pTFmOnR/VVbjJk4K2xE34GC2nAdUQkPFuyaFisicc6HRMOYXPuF0DuwITEKnjxgNj P+qDrh0db7PAjO1D4d5ftfrsf+kdRR4gKVGSk8Tz2WwvtLAroJM4nXjNPIBJNT4w/FWWc/5qPHJy 2U+eITZ5LLE5s45mX2oPFknWqxBobQZ8a9dsZ3dSPZBvE9ZrmtFLrVrN4eo1jsXgAp1+p7bkfqd3 BgBEmfsYWlBXO8rVXfvPgLs32VdVNZxb/CDWPqBsiYv0Hv3HPsz07j5b+/cVoWqyHDKzkaVbxfq/ 7auNVRmPB3v5SWEsH8xi4Bez2V9UKxfYCnqsjp8RaC2/khxKt0A552Eaxnz/4ly/2C7wkwTQnBmd lFYhAflWKQ03Ufiu8t3iBE3VJbc25oMrglj7TRZrmKq3CkbFnX0fyulB+kHimrt6PIWn7kgyl9ae lIl6vtbhMA+l0nfrsORMa4kobqQ5C5rveVgmcIad67EDa+UqEKy/GltUwlSh6xy+TrK1tzDvAgMB AAGjggHMMIIByDAfBgNVHSMEGDAWgBQJwPL8C9qU21/+K9+omULPyeCtADAdBgNVHQ4EFgQUzMeD Mcimo0oz8o1R1Nver3ZVpSkwDgYDVR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYw FAYIKwYBBQUHAwQGCCsGAQUFBwMCMEAGA1UdIAQ5MDcwNQYMKwYBBAGyMQECAQEBMCUwIwYIKwYB BQUHAgEWF2h0dHBzOi8vc2VjdGlnby5jb20vQ1BTMFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9j cmwuc2VjdGlnby5jb20vU2VjdGlnb1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1h aWxDQS5jcmwwgYoGCCsGAQUFBwEBBH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdv LmNvbS9TZWN0aWdvUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAj BggrBgEFBQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5m cmFkZWFkLm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAyW6MUir5dm495teKqAQjDJwuFCi35h4xgnQv Q/fzPXmtR9t54rpmI2TfyvcKgOXpqa7BGXNFfh1JsqexVkIqZP9uWB2J+uVMD+XZEs/KYNNX2PvI lSPrzIB4Z2wyIGQpaPLlYflrrVFKv9CjT2zdqvy2maK7HKOQRt3BiJbVG5lRiwbbygldcALEV9Ch WFfgSXvrWDZspnU3Gjw/rMHrGnqlHtlyebp3pf3fSS9kzQ1FVtVIDrL6eqhTwJxe+pXSMMqFiN0w hpBtXdyDjzBtQTaZJ7zTT/vlehc/tDuqZwGHm/YJy883Ll+GP3NvOkgaRGWEuYWJJ6hFCkXYjyR9 IzGCBMcwggTDAgEBMIGsMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVz dGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMT NVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEA xr4ZlmdAxAMdKFES+jupfjANBglghkgBZQMEAgEFAKCCAeswGAYJKoZIhvcNAQkDMQsGCSqGSIb3 DQEHATAcBgkqhkiG9w0BCQUxDxcNMjQxMjIzMTQyNDEwWjAvBgkqhkiG9w0BCQQxIgQgLjoGuIHB omRLJQwpN2tHqfuHO4jPYthyM3wmAiO5lnEwgb0GCSsGAQQBgjcQBDGBrzCBrDCBljELMAkGA1UE BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEYMBYG A1UEChMPU2VjdGlnbyBMaW1pdGVkMT4wPAYDVQQDEzVTZWN0aWdvIFJTQSBDbGllbnQgQXV0aGVu dGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIRAMa+GZZnQMQDHShREvo7qX4wgb8GCyqGSIb3 DQEJEAILMYGvoIGsMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVy MRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNl Y3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEAxr4Z lmdAxAMdKFES+jupfjANBgkqhkiG9w0BAQEFAASCAgB2ROMHAW1CTw8/toCS+JWYqa5n6wjd5xfn nZbcg7i3AUU6sBx9kqDWKI1YJeTYI1EdW4cPKSqdeHfidYyGLcJGaM+JeuvKK3xo5AKFQ2GGRzim yiWbUPm9aFa9bxCtDWUgxPrRk0xLlrGSRa7U6iJMGXy+U0PF1sJ1c7V6E8YYl9Rbw9LeDsMzjg6E hN0Yc2KO0bt/8hsGorjzZIQBWt+I0XLBX3MnfUs8BsKl1xBfer/T+36EHNW9BlWjbqPzLSzP8Bs4 g+R0rXMoZ61Uu1Qz9toBq1AlDv82qYRDJdaEEjxqLYAEHDAVljT3LP1zywDgJLAM36vgu3KJhW0n n98ica56J0VFZtQLh9DYPlPmVwzNL+aMMBXPqqaslAV9U0k9cqBhPSGrsGspq2AXaDaLy26EE4dd 7p/QFGkSmsnYmaWMc+UN1l8zEL9id9M5S33JEN0Y4sr0cfRpRl9lEBFyxBDpmwMIfV0nL4ZkHJf5 RnZsGIpFAJJQncMsEVw5KChK35+X7v3g23ek+G+Riu/JynTVc5Xd1lVn2PLp6+fGQzaPVnjEBdKI 2v+o7hD5/U0GRAGgHufY4RLnUMj7AddZ+HsfjHNNFrMPELbyUmpdCwPDLYK4sKEcmx5sOaEva0FB WUSQIRcflfFFqbC1SgAifWmi3HgorQl4lGllRd08/QAAAAAAAA== --=-2cuJts2lurvGX3hsXblo--