[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [linux-5.4 test] 174684: regressions - FAIL


  • To: Jan Beulich <jbeulich@xxxxxxxx>
  • From: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>
  • Date: Thu, 10 Nov 2022 08:54:06 +0000
  • Accept-language: en-GB, en-US
  • Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lAE9CgBaMSnFZYP3w6bTPL/eIphS78Wy40DT5C3H23E=; b=GSqDJkG+ktXEb1bzeIr5j89OXTn/P6zyMCTzFvsk6pGveHrj15sSqUJmFm0aTPretYLw3Jj1wj/RlDhXAkTTLAtqwtWVmocMuRKF/TYAaneBJhgOQbIeVzQsNQsmUQgvCuViuIoF88ESHeblyzKvWZHz5f1Kh9D7t+QhRzU51+H4k2YLHHhLoZXdw0jgswP42Ky140F0Lt4FZit50LFkfg7mSs2Q79JESODUdjJ4276WSecAFKUFha5ihZ08h4rXQxCY3dsdRxEUKFrPwe84pXs6cmG/T2pmvJtGzgMwzozUuaCjMgkW+YUVBKpzA7cCbuYTHTghIfG58eQHnDgR9g==
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lAE9CgBaMSnFZYP3w6bTPL/eIphS78Wy40DT5C3H23E=; b=ZH1TG8EnAWQeY9a+b5KBIKSBk89nZzbJ4oyfJ1Dfi/7l9ExGKjMLVQMYv4gn1ve3Zl9abdRlagxH1DGGMhRXGwNiuHPX1xAVQKZUtiDpk8hu7Nw83eh1KrJXx0vy65jGwqTFDV3hTXlvoMBV2x8e4YB7JrURBqw7zlNr30DpDov962p/ySamYIPvStjBMUd1oQPYBPYXXHgrtxs01Y734zEouvqO7fZsNXD3dQThBJyqQujnsWZlJGWYWQyLiMT/LdUt/CZtWw5TTgZQwGYlRbxjq+uU/rEYPpl5fGrCBRuSFr8aDtFuNEs+dfKEfIYkdPg7yvZqSML93BYBGbq9fA==
  • Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=F8UQG72crJZajTCLcrPvdckCLVJh/VT1mM1ftpVSQs6NbjIMr96YKIxqFTCalrk19WPAVV9toeOhzZYNy7B/KfGf6cfpqgd47cRgDoZEJemVI5Y0jKfmJ1qfieCY7dmBLFdCRSjffo27f0n28nBKHVQpldD14g57L+2rqWQaqUOxBj7pvXOix0IsT8mmRhFdJ5XFrOLj5pKn1/pqWlzHf4klvMcrEPG+gi4foabFi5PK5NR4nWYGhp3omIB9PZzBQHYJ60Cbr3nnKaPIkofDI3ca30MwJIz/cl7H63wwswlV2WE6a7olppzAmn4Hj+SncA8BraPN6tBaqRf421XFrw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=da/IsVPCbXZcfJcQL9GPXQjYndKbFFccSC11qGnyMolUpi8vnE4zPaBPd3Ehh9kOLOGcbbYtQzhkdj4DHg3DB3iGpQK4FDNhZlhpm36oYUgkCx0Jg6usDCSyAK0N5ygurQlHqTPbx7L9DFWK3UbxsIOgbRql9ZFRHaQsR+z849oTuV2FkLfH6SajrLH2RuOgfX0JF3xtk8gWbMJbYUd5JD8J/Xq/nE02PxEOpFqrPnnGwByaO/BG8Bgu3h3I1cyaWeqebV7JbqleF4PH4p//0FIr8ufCXcV+ku0UyaD2FrV4O4o/57PHX07i4mywkt8R/jI3hJGhMSJlaAxA4zrtYQ==
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Cc: Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Volodymyr Babchuk <volodymyr_babchuk@xxxxxxxx>, osstest service owner <osstest-admin@xxxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 10 Nov 2022 08:54:33 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHY9K76AziskhXElk+EohtHPDzhJa430IgAgAAKE4A=
  • Thread-topic: [linux-5.4 test] 174684: regressions - FAIL

Hi Jan,

> On 10 Nov 2022, at 08:18, Jan Beulich <jbeulich@xxxxxxxx> wrote:
> 
> On 10.11.2022 03:48, osstest service owner wrote:
>> flight 174684 linux-5.4 real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/174684/
>> 
>> Regressions :-(
>> 
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>> test-armhf-armhf-xl-credit2 18 guest-start/debian.repeat fail REGR. vs. 
>> 174540
> 
> This now looks to be failing relatively frequently (about every other flight),
> and I'd rather suspect it to point at a hypervisor issue than a Linux one.
> Looking at the log I found
> 
> (XEN) d5v0: vGICD: unhandled word write 0xffffffff to ICACTIVER0
> (XEN) arch/arm/traps.c:1985:d5v1 HSR=0x80000006 pc=00000000 gva=0 
> gpa=0000000000000000
> (XEN) arch/arm/traps.c:1985:d5v1 HSR=0x80000006 pc=0x00000c gva=0xc 
> gpa=0x0000000000000c
> (XEN) arch/arm/traps.c:1985:d5v1 HSR=0x80000006 pc=0x00000c gva=0xc 
> gpa=0x0000000000000c
> 
> with the last two messages then repeated over and over, many dozen times a
> second. Which makes me wonder whether that verbosity alone is causing a
> problem.

The 2 messages are not necessarily related.

The first one is a warning and the write is ignored, we see that all the time 
with Xen in Debug mode during linux boot.

The other traps, with PC at 0xc look like there was an exception in Linux but 
the register pointing to the exception table is at 0 which means the guest did 
an exception before it initialised the exception table pointer.

When a guest is going to this kind of error it will loop on this:
- exception
- jump to exception table at 0 + offset depending on exception
- trap in xen because 0 is not mapped and push back the exception to guest
- go back to step 1

Now it could be that the first exception we reported back to guest should not 
have been reported and the guest falls into this loop after.

Our CI was a bit late compared to staging status until today (due to some 
servers being down).
I will check what is happening on our side with the last status today on arm32.

Cheers

Bertrand

> 
> Jan




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.