[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [XenSummit 2017] Build tools follow up



Hi, all!

This is a follow-up on Xen distribution build systems we saw at the
summit and invitation for sharing thoughts and ways we build our images and
distros. I would like to specifically ask OpenXT project to reply with the
description of their build system so we have clear picture of what is being
developed and used around. And of course if there are other approaches to do the
same you are welcome to share those as well.

On our side at EPAM we have developed a distribution which is called xt-distro
(xt stays for Xen troops [0]) with the following goals in mind:
1. Make it possible to easily build different images as a single distribution
   - Separate images for different domains should be combined into a
     distribution, e.g. set of images/artifacts required to run them as a
     system
- Xt-distro allows doing so as easy as running bitbake with the predefined
     target, e.g. “bitbake xt-image”
2. Easily deal with different BSPs for different platforms using a unified way
   with little or no EPAM specific code/scripts
- We use bitbake and stripped version of Poky/meta-openembedded, so there are little in-house extensions we added to deal with Yocto based BSPs [1], [2] - We use bitbake’s well defined scripting language [3], so no yet another
     language to learn
- We use Google repo tool to maintain sets of meta layers as manifest files [4]
3. Ability to easily collect component revisions, so any build can be
   reproduced if need be
- We save commit IDs of every component of the build including versions of the
     meta layers
4. Ability to easily customize and tune builds, e.g. URIs, versions/commitIDs,
   branches used
   - This is purely done in bitbake’s recipe language
5. Make patching process easy
- With our extensions you can use bblayers.conf, local.conf, patches which
     are part of a meta layer which is usually a manual step
6. Code reuse is a must, e.g. the same set of software must be easily built for different platforms without copying build scripts of the existing components - With bitbake’s meta layers we define generic recipes, e.g. suitable for all
     platforms [5] (think of it as a library) and tuning meta layers [6]
(we use product concept here) which define specific versions/revisions of the
     components, apply specific patches and add/remove software
7. Cross compilation must be an easy task to do
- This is easily achieved with Yocto builds and usually not a problem for other
     build systems as well (with SDKs)
   - Allows building for x86/ARM and other architectures as well
8. There must be a simple way to share build artifacts of different domains between each other, for example, DomU’s kernel should be a part of Dom0’s rootfs, so
   xl/libxl can access it
   - This is achieved with bitbake’s recipes, so no EPAM extensions
9. Possibility to easily use different build systems for different components of
   the system.
- Bitbake allows you building makefile based projects, CMake etc. out of the box,
     so adding firmware, stubdoms made easy
10. Development support
- We use SDKs built by bitbake so during everyday development you are packed with all that is needed to re-create build environment (with both host and target
     environments, e.g. x86 host tools and ARM cross-compiler)
- Speed up builds for developers within organization, e.g. we reuse downloads and
     build cache between all of the developer’s machines on our network
11. Make the build system suitable for build automation
    - We have created a python script which can easily be used with
      Jenkins or even cron [7]

For example, with xt-distro our development product delivers: Xen image and
policy file, Dom0 kernel and dtb images (Dom0 + DomU), Dom0 rootfs image (with
embedded kernels and configuration for DomU), DomU rootfs image. This
effectively means, that the build system can provide you with the complete set
of images to run your product/distribution.

We are looking forward for any kind of feedback and will be glad to collaborate
on the above.

Thank you,
Xen-troops at EPAM

[0] https://github.com/xen-troops/xt-distro
[1] https://git.yoctoproject.org/cgit/cgit.cgi/poky
[2] http://cgit.openembedded.org/meta-openembedded
[3] http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html
[4] https://gerrit.googlesource.com/git-repo
[5] https://github.com/xen-troops/meta-xt-images
[6] https://github.com/xen-troops/meta-xt-prod-devel
[7] https://github.com/xen-troops/build-scripts


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.