Re: [Xen-devel] [xen-unstable test] 120287: regressions - FAIL

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 08.03.18 at 23:24, <osstest-admin@xxxxxxxxxxxxxx> wrote:
> flight 120287 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/120287/ 
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-i386-migrupgrade  11 xen-boot/dst_host        fail REGR. vs. 
> 120037
>  test-amd64-i386-livepatch     7 xen-boot                 fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-boot     fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemuu-win10-i386  7 xen-boot          fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemuu-ovmf-amd64 8 host-ping-check-xen fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-boot     fail REGR. vs. 
> 120037
>  test-amd64-i386-libvirt       7 xen-boot                 fail REGR. vs. 
> 120037
>  test-amd64-i386-rumprun-i386  7 xen-boot                 fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  7 xen-boot fail REGR. vs. 
> 120037
>  test-amd64-i386-libvirt-xsm   8 host-ping-check-xen      fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemuu-win7-amd64  7 xen-boot          fail REGR. vs. 
> 120037
>  test-amd64-i386-freebsd10-amd64  7 xen-boot              fail REGR. vs. 
> 120037
>  test-amd64-i386-examine       8 reboot                   fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-xsm        7 xen-boot                 fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. 
> vs. 120037
>  test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-boot          fail REGR. vs. 
> 120037
>  test-amd64-i386-libvirt-pair 10 xen-boot/src_host        fail REGR. vs. 
> 120037
>  test-amd64-i386-libvirt-pair 11 xen-boot/dst_host        fail REGR. vs. 
> 120037
>  test-amd64-i386-xl            7 xen-boot                 fail REGR. vs. 
> 120037
>  test-amd64-i386-qemut-rhel6hvm-intel  7 xen-boot         fail REGR. vs. 
> 120037
>  test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-boot         fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemut-ws16-amd64  7 xen-boot          fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-raw        7 xen-boot                 fail REGR. vs. 
> 120037
>  test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  7 xen-boot fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemut-win10-i386  7 xen-boot          fail REGR. vs. 
> 120037
>  test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-boot           fail REGR. vs. 
> 120037
>  test-amd64-i386-qemut-rhel6hvm-amd  7 xen-boot           fail REGR. vs. 
> 120037
>  test-amd64-i386-xl-qemut-win7-amd64  7 xen-boot          fail REGR. vs. 
> 120037
>  test-amd64-i386-freebsd10-i386  7 xen-boot               fail REGR. vs. 
> 120037
>  test-amd64-i386-pair         10 xen-boot/src_host        fail REGR. vs. 
> 120037
>  test-amd64-i386-pair         11 xen-boot/dst_host        fail REGR. vs. 
> 120037

So something between 7bf61602f295676c8b0ff61e4c584fc2bd57e4cf
(flight 120253) and 31bf55cb5fe3796cf6a4efbcfc0a9418bb1c783f (this
flight) must have broken 32-bit Dom0 across the board, without me
being able to spot anything in the first log I've looked at (the prompt
isn't reached, and the pings remain unanswered).

In the second log (test-amd64-i386-libvirt-xsm), however, there is:

Mar  8 09:46:27.227176 [  353.928122] kernel BUG at fs/ext4/inode.c:2616!

Mar  8 09:46:27.235145 [  353.928128] invalid opcode: 0000 [#1] SMP
Mar  8 09:46:27.235184 [  353.928143] Modules linked in: xen_acpi_processor 
xen_gntalloc ext4 jbd2 mbcache igb i2c_algo_bit
Mar  8 09:46:27.251106 [  353.928162] CPU: 7 PID: 1161 Comm: kworker/u16:9 Not 
tainted 4.9.84+ #1
Mar  8 09:46:27.259131 [  353.928169] Hardware name: Intel Corporation 
S1200RP/S1200RP, BIOS S1200RP.86B.03.01.0002.041520151123 04/15/2015
Mar  8 09:46:27.267101 [  353.928183] Workqueue: writeback wb_workfn 
(flush-253:0)
Mar  8 09:46:27.275136 [  353.928190] task: dacb2900 task.stack: c4b46000
Mar  8 09:46:27.283136 [  353.928196] EIP: 0061:[<e1332640>] EFLAGS: 00210246 
CPU: 7
Mar  8 09:46:27.283175 [  353.928209] EIP is at 
mpage_prepare_extent_to_map+0x220/0x250 [ext4]
Mar  8 09:46:27.291138 [  353.928216] EAX: 00000000 EBX: dfba8ea0 ECX: 00000000 
EDX: 40010079
Mar  8 09:46:27.299120 [  353.928222] ESI: 00000000 EDI: c4b47d48 EBP: c4b47cd8 
ESP: c4b47c68
Mar  8 09:46:27.307116 [  353.928229]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 
0069
Mar  8 09:46:27.315061 [  353.928239] CR0: 80050033 CR2: b7577740 CR3: 1a247000 
CR4: 00042660
Mar  8 09:46:27.323055 [  353.928250] Stack:
Mar  8 09:46:27.323085 [  353.928253]  00000000 0000000e 00000000 00000000 
c41111ac 00000001 ffffffff 00000c00
Mar  8 09:46:27.331086 [  353.928268]  00000001 00000001 00000000 dfba8ea0 
00003a9a 00000015 db722c00 db722c00
Mar  8 09:46:27.339080 [  353.928283]  00000002 c4b47cd8 e13646c7 02400040 
00000002 00000ace 00000002 00000ace
Mar  8 09:46:27.347077 [  353.928297] Call Trace:
Mar  8 09:46:27.355050 [  353.928316]  [<e13646c7>] ? 
__ext4_journal_start_sb+0x57/0xc0 [ext4]
Mar  8 09:46:27.363051 [  353.928357]  [<e1336e67>] ? 
ext4_writepages+0x447/0x760 [ext4]
Mar  8 09:46:27.371052 [  353.928376]  [<e1336e90>] ext4_writepages+0x470/0x760 
[ext4]
Mar  8 09:46:27.371091 [  353.928385]  [<c133bdba>] ? find_next_bit+0x1a/0x30
Mar  8 09:46:27.379049 [  353.928392]  [<c1326929>] ? cpumask_next_and+0x29/0x40
Mar  8 09:46:27.387069 [  353.928399]  [<c10fe8b4>] ? 
find_busiest_group+0x24/0x4f0
Mar  8 09:46:27.395116 [  353.928406]  [<c1107c99>] ? 
__raw_callee_save___pv_queued_spin_unlock+0x9/0x10
Mar  8 09:46:27.403084 [  353.928415]  [<c11a4688>] do_writepages+0x18/0x30
Mar  8 09:46:27.403120 [  353.928421]  [<c1215af5>] 
__writeback_single_inode+0x35/0x340
Mar  8 09:46:27.411089 [  353.928430]  [<c17ffa17>] ? 
_raw_spin_unlock_irqrestore+0x27/0x40
Mar  8 09:46:27.419057 [  353.928437]  [<c121628d>] 
writeback_sb_inodes+0x20d/0x420
Mar  8 09:46:27.427045 [  353.928444]  [<c121651c>] 
__writeback_inodes_wb+0x7c/0xb0
Mar  8 09:46:27.435088 [  353.928450]  [<c121677a>] wb_writeback+0x22a/0x2c0
Mar  8 09:46:27.435125 [  353.928456]  [<c1216d17>] wb_workfn+0x1c7/0x3a0
Mar  8 09:46:27.443062 [  353.928463]  [<c10ea762>] ? 
finish_task_switch+0x82/0x240
Mar  8 09:46:27.451051 [  353.928469]  [<c1806105>] ? __clear_rsb+0xd/0x32
Mar  8 09:46:27.451084 [  353.928476]  [<c10ddce9>] process_one_work+0x129/0x3c0
Mar  8 09:46:27.459078 [  353.928482]  [<c10de382>] worker_thread+0x102/0x4f0
Mar  8 09:46:27.467069 [  353.928488]  [<c10de280>] ? 
mod_delayed_work_on+0x70/0x70
Mar  8 09:46:27.475053 [  353.928494]  [<c10e3044>] kthread+0xb4/0xd0
Mar  8 09:46:27.475087 [  353.928500]  [<c10881d1>] ? __switch_to+0x201/0x640
Mar  8 09:46:27.483108 [  353.928506]  [<c10e2f90>] ? kthread_park+0x50/0x50
Mar  8 09:46:27.491074 [  353.928512]  [<c17ffd50>] ret_from_fork+0x30/0x3c
Mar  8 09:46:27.491129 [  353.928517] Code: b0 39 55 a8 0f 83 71 fe ff ff 31 db 
eb c7 90 8d 74 26 00 89 d8 e8 e1 52 e6 df e9 e8 fe ff ff 8d 74 26 00 0f 0b 8d 
b6 00 00 00 00 <0f> 0b 8d b6 00 00 00 00 8d 45 b4 e8 e0 45 e7 df 83 c4 64 89 d8
Mar  8 09:46:27.515121 [  353.928613] EIP: [<e1332640>] [  353.928620] 
mpage_prepare_extent_to_map+0x220/0x250 [ext4]
Mar  8 09:46:27.523224 [  353.928626]  SS:ESP 0069:c4b47c68

But that doesn't tell me anything.

Most of the commits in that range are mine, yet I can't spot
anything that would look like it might affect _only_ 32-bit Dom0.
Unless anyone else has an idea, I'm afraid we'll have to wait
for the bisector to pinpoint the bad commit.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel