Re: [PATCH v1 02/11] ci: increase timeout for hw tests

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Thu, Apr 03, 2025 at 01:32:38PM +0200, Jan Beulich wrote:
> On 03.04.2025 13:04, Marek Marczykowski-Górecki wrote:
> > It appears as sometimes it takes more time for Xen even start booting,
> > mostly due to firmware and fetching large boot files by grub. In some
> > jobs the current timeout is pretty close to the actual time needed, and
> > sometimes (rarely for now) test fails due to timeout expiring in the
> > middle of dom0 booting. This will be happening more often if the
> > initramfs will grow (and with more complex tests).
> 
> With that, ...
> 
> > This has been observed on some dom0pvh-hvm jobs, at least on runners hw3
> > and hw11.
> > 
> > Increase the timeout by yet another 60s (up to 180s now).
> 
> ... is this little a bump going to be sufficient? How about moving straight
> to 5min?

I don't like this, as many (most) actual failures are visible as timeout
(for example panic that prevents reaching Alpine prompt). One
improvement I can see is splitting this into two separate timeouts: one
before seeing the first line from Xen and then the second one for
reaching Alpine login prompt. The first one can be longer as its mostly
about firmware+fetching boot files and shouldn't hit on crashes (unless
a crash happen before printing anything on the console - but those are
rare).

> As to observed failing jobs - the PV Dom0 boot failure seen today looks to
> also be due to too short a timeout.

As responded on Matrix, I'm not so sure, there is over 1m wait after
"Built 1 zonelists, mobility grouping on.  Total pages: 8228487" line
from dom0 (or a bit later, due to buffering by sed), while in successful
test next lines follow instantaneously.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature