[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test


  • To: Stefano Stabellini <sstabellini@xxxxxxxxxx>
  • From: Anthony PERARD <anthony.perard@xxxxxxxxxx>
  • Date: Wed, 27 Oct 2021 14:59:22 +0100
  • Authentication-results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none
  • Cc: <xen-devel@xxxxxxxxxxxxxxxxxxxx>, <iwj@xxxxxxxxxxxxxx>, <cardoe@xxxxxxxxxx>, <wl@xxxxxxx>, <andrew.cooper3@xxxxxxxxxx>, "Stefano Stabellini" <stefano.stabellini@xxxxxxxxxx>
  • Delivery-date: Wed, 27 Oct 2021 14:00:00 +0000
  • Ironport-data: A9a23:1y4VFaDTeu1wdxVW/7Xkw5YqxClBgxIJ4kV8jS/XYbTApDx30GNRy GAcUD/SPvqDYDekf9oiao3i9EMPvJWGzII3QQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WLGs1hxZH1c+EX550007wobVv6Yz6TSHK1LV0 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/tCe7kfBsw sV06N+6dEAxNZyUxtwfekwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr KFGbmBWBvyAr7veLLaTQ+9whsMlPY/zMZkWoH1IxjDFF/c2B5vERs0m4PcFjG1g15sURZ4yY eJBbiRfSiWdbydCYHkbIdVgg8OmqVXwJmgwRFW9+vNsvjm7IBZK+LPkKtbSd/SBTN9ZmUver WXDl0zmBjkKOdrZziCKmlq3muLBlCX8HpkOHbe18vprhly71m0XCRsGE1C8pJGRkVWiUthSL 0gV/CsGrqUo8kGvCN7nUHWQsHOC+xIRRddUO+k78x2WjLrZ5R6DAWoJRSIHb8Yp3Oc3Wj4Cx lKPh8nuBzFkrPuSU331y1uPhWrsY25PdzZEPHJaC1teizX+nG0tpi3dfNhDDaSlt4C2Ewy32 zu79HYGhZxG2KbnyJ6H1VzAhjutoL3AQQg0+hjbUwqZ0+9pWGK2T9f3sQaDvJ6sOK7cFwPb5 CFVxKBy+chXVcnV/BFhVtnhC11ACxytCzbbnUJ0V6co8zCg6hZPlqgBvWkgei+F3isCEAIFg XM/WysNu/e/31PwNMebhr5d7exwlcAM8vy+D5jpgiJmOMQZSeN+1HgGibSs927silMwtqo0J I2Wd82hZV5DV/86lmXtF71FiOd2rszb+Y81bcqjp/hA+eHHDEN5tJ9faAfeBgzHxPrcyOkqz zqvH5TTkEgOOAEPSiLW7ZQSPTg3wYsTXvjLRzhsXrfbeGJOQTh5Y9eImO9JU9E1zsx9y7aTl lngCxAw9bYKrSCeQel8Qis4M+2HsFcWhS9TABHAyn7yhCd6Mdf1tf5OH3b1FJF+nNFeITdPZ 6FtU6297j5nE2uvF+01YcavoYp8Wg6sgA7SbSOpbCJmJ8xrRhDT+8+idQzqrXFcAi2yvMo4g ruhygKEHsZTG1U8VJ7bOKC10le8nXkBg+YuDUHGFcZeJRf3+49wJi2v0vJue5MQKQ/Ozyex3 hqNBUtKvvHEpoI4qYGbha2No4qzPfF5G05WQzvS4bqsbHGI9Wu/245QFu2PeGmFBm/z/ayjY 8RTzu39b6JbzAob7dIkHu8yn6wk5tbpq7tL9SheHS3GPwaxF7dtAniaxs0T5KdD8aBU5FmtU UWV99gEZbjQYJH5EEQcLRYOZ/iY0a1GgSHb6Pk4LRmo5CJz+7bbA0xeMwPV1X5YJbpxdogk3 f0gqIgd7Anm0kgmNdOPjyZ18WWQLyNfD/V7589CWIK72BA2zlxiYIDHDn6k6Z6CXNxAL002L 2LGn6HFnbldmhLPfndb+aIhBgaBaUDiYCx38WI=
  • Ironport-hdrordr: A9a23:+5L9t6rDppiilzssayX6aWYaV5oreYIsimQD101hICG8cqSj+f xGuM5rsSMc6QxhPU3I9ursBEDtex/hHNtOkO4s1NSZLWvbUQmTTL2KhLGKq1aLJ8S9zJ8/6U 4JSdkZNDSaNzlHZKjBjzWFLw==
  • Ironport-sdr: 2tUkm5hz3JYdCPGIFSIijTldf9nl0hyIjRtlnGloDs3Ccg7iAbOpWXORJLbKlb7ZV3QJ2aDipV U+iME93b8WPCnGFArq4vYqg9wIz3aWKcA5DOXcNgAfCsdu30aCuLIPO4mUZr9ABMcROe3bei62 +U+fJwNopW+YR46K+jgACeLDLRJH+tMK0cwYKusM0Xe411/ESrdsItmJ9qodyXAkfpwvgS6yY9 shtLsaS0LNOzc4CsLFYUtGmxyCB4PGQP2ks8GbkRYB+3sZFboIS3P6OVlhxor4fJMTz5voBmS3 Pzem3vCiilzH7LzOyMk5NaZA
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Mon, Oct 25, 2021 at 06:33:53PM -0700, Stefano Stabellini wrote:
> On Mon, 25 Oct 2021, Anthony PERARD wrote:
> > There is something I'm missing, how is it a problem to have a container
> > that is a bit bigger? What sort of problem could we have to deal with?
> 
> It takes time to clone the container in the gitlab-ci, the bigger the
> container the more time it takes. It is fetched over the network. Now we
> are fetching qemu (as part of the container) 10 times during the build
> although it is not needed.

I guess the issue would be more like we don't do enough caching with our
gitlab runners. So package installation is just a workaround.

> But we do have a severe problem at the moment with external sources: our
> "git clones" keep failing during the build on x86. That is definitely
> something worth improving (see my other email thread on the subject) and
> it is the main problem affecting gitlab-ci at the moment, I keep having
> to restart jobs almost daily to get the overall pipeline to "pass".
> 
> If you have any ideas on how to stop fetching things using "git" from
> external repositories in gitlab-ci that would be fantastic :-)
> The only thing I could think of to "fix it" is moving all external repos
> to gitlab repositories mirrors.

I don't think that would work, I've seen the initial clone/fetch of a
job fail as well, so from gitlab. If we could have a cache of those
external resources closer to the runners, that would be better.

> > > I am not entirely sure what is the best solution overall, but for this
> > > series at this stage I would prefer to keep the same strategy used for
> > > the ARM tests (i.e. reuse the debian unstable build container and
> > > apt-get the few missing packages.) If we do change the way we do it, I
> > > would rather change both x86 and ARM at the same time.
> > 
> > I'm pretty sure the best strategy would be to do as little as possible
> > during a job, download as little as possible and possibly cache as much
> > as possible and do as much as possible ahead of time. Feel free to
> > change the Arm test, but I don't think it is necessary to change the Arm
> > test at the same time as introducing an x86 test.
> 
> I agree.
> 
> At the same time it would be nice to follow the same strategy between
> x86 and ARM going forward: if one optimization is made for one, it is
> also made for the other.

Probably better, yes.

Thanks,

-- 
Anthony PERARD



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.