[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xen-unstable test] 169819: regressions - FAIL

  • To: Jan Beulich <jbeulich@xxxxxxxx>
  • From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Date: Fri, 29 Apr 2022 13:00:48 +0200
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z9JDzMPrfVicYGsXcEMPT0UKo3t69Qo+iV+CC3lJ88Y=; b=B4p3UXE8Q5eg+0K0WtLH0sDySgIRLlPuGB0wmm3NYkPUDT+mbtasSRXlsgsW1Rd88o7+rCSm9uEEHAcwLkhgK6atAVM1iBHKuwXlahK+Tb+vkXlK3RfBJv7jlI6Vynyjj9s0PpX7imZZNj0hsJuVX4zJTNUtYCcKzgBDjlgXKHnjoFThyj+vCQV7DTVKvPBrn/80cOjgBcv6omxQUaymMUOgC6OYc8uh7awNz5ZTmKw0zzdwbFqSo+JFMK10sFhWHF2jAmmdLbPh7y/Bp03DKk8VgMYkkH00574pu27Cfp/L/RdVknBmCsJpCnWFbBRF3K5kFPHLDEv1f3s/M8yQLg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cV86LSp1n+Wut6d3N3d+8tHPOK2YAaU148iu+Ust8oJiPn/rYtkDkBZ4htQnIzYR1fEeWw0cKTNUkurmQxCdttjuM618CsPnfAvUpK6If2ftD1rFmjtUb28JlXKBRUeADZ479NbQ2P4t22xowWOVzO5SoV1Nx+3QCJCHHN9r1wKVj0Utdr0THjzoXhm/NIGpSpS1pCFVdylJYnvOEchjpBYMvUWuqubZbibUG21aRAcXzhiIfg38ml9EHk/Kidmlue7c3AYyvS63ZKi8BMuRvOOwAYynLN1/KEI5PA/DNKbxEZ6jkf/OavHSWZ/QQAM/S1RomPqab57R0+pbJyF/DQ==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx, Juergen Gross <jgross@xxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>, osstest service owner <osstest-admin@xxxxxxxxxxxxxx>
  • Delivery-date: Fri, 29 Apr 2022 11:01:02 +0000
  • Ironport-data: A9a23:zMWEx682LWq4xQPMRFM5DrUDqn+TJUtcMsCJ2f8bNWPcYEJGY0x3y GsXDW6Abqzfa2T2ftkga4/i8koPu5ODmoJiGVA6rn08E34SpcT7XtnIdU2Y0wF+jyHgoOCLy +1EN7Es+ehtFie0Si+Fa+Sn9T8mvU2xbuKU5NTsY0idfic5DnZ44f5fs7Rh2NQw3YLoW1nlV e7a+KUzBnf0g1aYDUpMg06zgEsHUCPa4W5wUvQWPJinjXeG/5UnJMt3yZKZdhMUdrJ8DO+iL 9sv+Znilo/vE7XBPfv++lrzWhVirrc/pmFigFIOM0SpqkAqSiDfTs/XnRfTAKtao2zhojx/9 DlCnYbzUgQKPKDlpMoiChMbUB5xZ6dZ3qCSdBBTseTLp6HHW13F5q00SWsQZMgf8OsxBnxS/ /sFLjxLdgqEm++93LO8TK9rm9gnK87oeogYvxmMzxmAVapgHc+FHvyMuYcwMDQY36iiGd7EY MUUc3x3ZQnoaBxTIFYHTpk5mY9Eg1GgK2QA8gLL+sLb5UDOyD1t07XJauH/IOKDX+lIoUSCi nv/qjGR7hYycYb3JSC+2nCmi/LLnCj7cJkPD7D+/flv6HWDy2pWBBAIWF+TpfiillX4S99ZM 1YT+Cclse417kPDZtvyRRqju1afowURHdFXFoUS4gaXy7GS5B2FHG8aVT1QQNs8vcQySHoh0 Vrht9TyHiBiq6HQQH6Y97G8vT60ITISL2IJeWkDVwRt3jX4iIQ6jxaKRNA6Fqew1oLxAWuon 2jMqzUijbIOi8JNz7+84V3MnzOroN7OUxIx4QLUGGmi62uVebKYWmBh0nCDhd4oEWpTZgDpU KQs8yRG0N0zMA==
  • Ironport-hdrordr: A9a23:1xkBdqkbMl0CNyYf4/rNpEGRuvPpDfOlimdD5ihNYBxZY6Wkfp +V8cjzhCWftN9OYhodcLC7V5Voj0mskKKdxbNhRYtKOzOWw1dATbsSlLcKpgeNJ8SQzI5gPM tbAstD4ZjLfCJHZKXBkXaF+rQbsb66GcmT7I+xrkuFDzsaDZ2Ihz0JdjpzeXcGIDWua6BJdq Z1saF81kedkDksH4yGL0hAe9KGi8zAlZrgbxJDLxk76DOWhTftzLLhCRCX0joXTjsKmN4ZgC L4uj28wp/mn+Cwyxfa2WOWx5NKmOH5wt8GIMCXkMAaJhjllw7tToV8XL+puiwzvYiUmR0Xue iJhy1lE9V46nvXcG3wiRzx2zP42DJr0HPmwU/wuwqVnSRULAhKePZptMZ8SF/0+kAgtNZz3O ZgxGSCradaChvGgWDU+8XIfwsCrDv7nVMS1cooy1BPW4oXb7Fc6aYF+llOLZsGFCXmrKg6De hVCt3G7vo+SyLUU5nghBgu/DWQZAVxIv/fKXJy+PB9kgIm0EyR9nFohfD2xRw7hdcAo5ot3Z WxDk0nrsALciYsV9MNOA4we7rINoXze2O9DIuzGyWQKEhVAQOFl3bIiI9Flt2CSdgv8KYYvq jnfRdxiVMSEniefPFmmqc7vyzwfA==
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
On Fri, Apr 29, 2022 at 12:37:13PM +0200, Jan Beulich wrote:
> On 29.04.2022 12:30, Roger Pau Monné wrote:
> > On Fri, Apr 29, 2022 at 07:46:47AM +0000, osstest service owner wrote:
> >> flight 169819 xen-unstable real [real]
> >> flight 169843 xen-unstable real-retest [real]
> >> http://logs.test-lab.xenproject.org/osstest/logs/169819/
> >> http://logs.test-lab.xenproject.org/osstest/logs/169843/
> >>
> >> Regressions :-(
> >>
> >> Tests which did not succeed and are blocking,
> >> including tests which could not be run:
> >>  test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 
> >> 169775
> >>  test-arm64-arm64-libvirt-xsm  8 xen-boot                 fail REGR. vs. 
> >> 169775
> >>  test-arm64-arm64-libvirt-raw  8 xen-boot                 fail REGR. vs. 
> >> 169775
> >>  test-arm64-arm64-xl-credit1   8 xen-boot                 fail REGR. vs. 
> >> 169775
> >>  test-arm64-arm64-xl-thunderx  8 xen-boot                 fail REGR. vs. 
> >> 169775
> >>  test-arm64-arm64-xl           8 xen-boot                 fail REGR. vs. 
> >> 169775
> >>
> >> Tests which are failing intermittently (not blocking):
> >>  test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail 
> >> pass in 169843-retest
> > 
> > Looked into this one, and it's slightly concerning, guest seems to be
> > stuck at installation:
> > 
> > Select and install software  [  481.093857] watchdog: BUG: soft lockup - 
> > CPU#1 stuck for 23s! [ksoftirqd/1:17]
> > [  509.093865] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [  545.093820] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  573.093809] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  609.093855] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [  637.093836] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [  673.093957] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  701.093854] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  733.093805] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  761.093817] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  797.093898] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [  825.093863] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  861.093865] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  889.093945] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [  925.093974] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  953.093925] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [  985.093832] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1013.093855] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1049.094031] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1077.093860] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1113.093938] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1141.093803] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1177.094051] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1205.093805] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1237.093955] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1265.094004] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1301.093835] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1329.094039] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1365.093883] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1393.094167] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1429.093857] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1457.093900] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1489.094026] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1517.093997] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1553.093996] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1581.094064] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1617.094076] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1645.093882] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1681.093896] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1709.094022] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1741.093870] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1769.093854] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1805.094017] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1833.093837] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1869.094043] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1897.094101] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1933.093879] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 1961.093933] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 1997.093952] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2025.093925] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2057.093895] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2085.094172] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2121.094018] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2149.094021] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2185.093931] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2213.093864] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2249.093951] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2277.093899] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2309.094054] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2337.093956] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2373.094111] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2401.094132] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2437.094064] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2465.094003] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2501.093959] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2529.094050] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2561.094027] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2589.094061] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2625.093999] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2653.093956] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2689.094024] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2717.094093] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2753.093913] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2781.094095] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2813.093959] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2841.094117] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2877.094041] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2905.094009] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 2941.093893] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 2969.094085] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
> > [ksoftirqd/1:17]
> > [ 3005.094061] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 3033.093917] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > [ 3065.094060] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
> > [ksoftirqd/1:17]
> > 
> > debina1 is one of the boxes I've put into production recently.
> 
> ISTR this issue having surfaced randomly before. With the randomness making
> it difficult to investigate.

Indeed, it seems to happen on other boxes also:

http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemut-debianhvm-i386-xsm/xen-unstable.html

At least doesn't seem to be explicitly related to debinas.

Thanks, Roger.

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.