Re: [Xen-devel] [OSSTEST PATCH 1/9] README.dev: Document the blessings

[Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>]

Ian Campbell writes ("Re: [OSSTEST PATCH 1/9] README.dev: Document the 
blessings"):
> On Thu, 2015-12-17 at 17:06 +0000, Ian Jackson wrote:
> > +Each flight has a `blessing' and an `intended blessing'.  The
> > +`intended blessing' is what the flight is going to be blessed as when
> > +its execution has completed.  The intended blessing controls host
> > +allocation.
> 
> I think this could also usefully mention that `blessing' is the current
> blessing, which may go through several phases during the life of the
> flight, hopefully culminating in being set to `intended blessing' if the
> flight is successful and that the archaeology tools only look at `blessing'
> and never `intended blessing'

Yes.

> > +(Normally the `intended blessing' is the same as the bless argument to
> > +sg-execute-flight aka the -B argument to mg-execute-flight.)
> 
> How does this interact with the intended blessing passed to cs-flight-
> create?

So actually there a flight has three blessings.

> > + * `real-bisect' and `adhoc-bisect': These are found only as the
> > +   blessing of finished flights.  (This is achieved by passing *-adhoc
> > +   to sg-execute-flight.)  This allows the archaeologist tools to
> > +   distinguish full flights from bisection steps.
> 
> I don't follow the reference to *-adhoc here, since it matches neither
> `real-bisect' nor `adhoc-bisect'.

`Passing *-adhoc' should read `Passing *-bisect'.

> > +There is a special exception to the tools' flight status checks: any
> > +flight whose blessing contains `play' can be operated on out of order.
> > +
> > +Flights blessings can be manually changed with cs-flight-bless.  Eg
> > +  ./cs-flight-bless FLIGHT broken-real real
> > +updates FLIGHT to be marked `broken' rather than `real'.  This can be
> 
> The example says `broken-real' but the text says `broken' (if the text
> didn't quote the `broken' it probably wouldn't matter.

I have clarified this.

> Maybe mention the need to pass the current blessing as a safety catch,
> since it is part of the example command?

Yes.

squash! patch and complete v2, below.

Ian.

From 73903500006c0451cc81697872096fe2d6fe02ab Mon Sep 17 00:00:00 2001
From: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Date: Thu, 17 Dec 2015 17:58:41 +0000
Subject: [OSSTEST PATCH] squash! README.dev: Document the blessings

v2: Improvements from review
---
 README.dev |   39 ++++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/README.dev b/README.dev
index 879af60..351cd25 100644
--- a/README.dev
+++ b/README.dev
@@ -252,13 +252,17 @@ Blessings are used:
    flight.  (Flights with blessing `foo' use only hosts which have
    flag `blessed-foo'.)
 
-Each flight has a `blessing' and an `intended blessing'.  The
-`intended blessing' is what the flight is going to be blessed as when
-its execution has completed.  The intended blessing controls host
-allocation.
-
-(Normally the `intended blessing' is the same as the bless argument to
-sg-execute-flight aka the -B argument to mg-execute-flight.)
+Each flight has a `blessing' (flights.blessing in the DB) and an
+`intended blessing' (flights.intended).  The `blessing' changes as the
+flight goes through the stages of its lifecycle.  The `intended
+blessing' is roughly what the flight is going to be blessed as when
+its execution has completed, and controls host allocation.
+
+There is also the blessing which the flight will actually get when it
+is finished, which is not represented in the DB; rather, it is the
+blessing argument to sg-execute-flight (aka the -B argument to
+mg-execute-flight).  Normally this is the same as the intended
+blessing in the DB, except for bisection flights.
 
 These are the principal (intended) blessings:
 
@@ -297,9 +301,9 @@ These are the principal (intended) blessings:
    when the hosts are ready.
 
  * `real-bisect' and `adhoc-bisect': These are found only as the
-   blessing of finished flights.  (This is achieved by passing *-adhoc
-   to sg-execute-flight.)  This allows the archaeologist tools to
-   distinguish full flights from bisection steps.
+   blessing of finished flights.  (This is achieved by passing
+   *-bisect to sg-execute-flight.)  This allows the archaeologist
+   tools to distinguish full flights from bisection steps.
 
    The corresponding intended blessing (as found in the `intended'
    column of the flights table) is `real'.  So the hosts used by the
@@ -320,6 +324,16 @@ There are also blessings for unfinished flights:
 
  * `broken': Something went catastrophically wrong.
 
+ * `broken-*': Conventionally set by hand when a flight needs to be
+   `un-blessed', perhaps because it contains data misleading to the
+   archaeologists.
+
+   A flight's blessings can be manually changed with cs-flight-bless:
+      ./cs-flight-bless FLIGHT broken-real real
+   updates FLIGHT to be marked `broken' rather than `real'.
+   (cs-flight-bless matches the existing blessing against its final
+   argument, to avoid accidents.)
+
 Note that osstest is generally `crash-only software': nothing will
 `clean up' a crashed flight and set its blessing to `broken'.  Flights
 which were in progress once upon a time but never completed, because
@@ -328,8 +342,3 @@ time.
 
 There is a special exception to the tools' flight status checks: any
 flight whose blessing contains `play' can be operated on out of order.
-
-Flights blessings can be manually changed with cs-flight-bless.  Eg
-  ./cs-flight-bless FLIGHT broken-real real
-updates FLIGHT to be marked `broken' rather than `real'.  This can be
-useful if a flight's data is misleading to the archaeologists.
-- 
1.7.10.4



From d677a8bfe576b44d28df4d5d62cbe9dec9fd5578 Mon Sep 17 00:00:00 2001
From: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Date: Thu, 17 Dec 2015 16:51:17 +0000
Subject: [OSSTEST PATCH] README.dev: Document the blessings

Signed-off-by: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>
---
v2: Improvements from review
---
 README.dev |  110 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)

diff --git a/README.dev b/README.dev
index 65ec111..351cd25 100644
--- a/README.dev
+++ b/README.dev
@@ -232,3 +232,113 @@ probably are after a reboot) by looking for processes 
relating to the
 lists flight,job combinations and then:
 
 $ ./mg-hosts previoustasks --clear
+
+
+Flight blessings and the blessed-* host flags
+=============================================
+
+Both flights and hosts have a `blessing'.
+
+Blessings are used:
+
+ * To avoid accidentally tools operating on flights which are in the
+   wrong phase of their existence.
+
+ * To control which automatic archaeologists will look at which
+   flights.  (Archeaology tools take arguments to control which
+   blessings they will consider.)
+
+ * To control which hosts are considered suitable for use with which
+   flight.  (Flights with blessing `foo' use only hosts which have
+   flag `blessed-foo'.)
+
+Each flight has a `blessing' (flights.blessing in the DB) and an
+`intended blessing' (flights.intended).  The `blessing' changes as the
+flight goes through the stages of its lifecycle.  The `intended
+blessing' is roughly what the flight is going to be blessed as when
+its execution has completed, and controls host allocation.
+
+There is also the blessing which the flight will actually get when it
+is finished, which is not represented in the DB; rather, it is the
+blessing argument to sg-execute-flight (aka the -B argument to
+mg-execute-flight).  Normally this is the same as the intended
+blessing in the DB, except for bisection flights.
+
+These are the principal (intended) blessings:
+
+ * `real': Flights which are fully production runs.  They use only
+   clean working trees with no `funny' environment variables, and
+   versions of osstest intended for production.  The data from such
+   flights is used, where applicable, to control production push gate
+   decisions, etc.
+
+   `blessed-real' hosts are fully production-ready.
+
+ * `play': Playground.  Flights may use weird software, dirty working
+   trees, strange environment variables, and so on.  It does not make
+   sense to point archaeology tools at such flights.
+
+   `blessed-play' hosts may be partially or completely nonfunctional,
+   or strange in some way.  For this reason `play' flights should not
+   normally use the automatic host allocator.
+
+ * `adhoc': Special-purpose flights.  They should be run with software
+   nearly as clean as `real': the version of the software used may not
+   be a branch intended for-production, but any enhancements or
+   modifications should not leave false or misleading information in
+   the history database.
+
+   `blessed-adhoc' hosts are just as good as `blessed-real' ones.
+
+   The `adhoc' blessing exists mostly to protect the main production
+   workflow from unintentional violations (for example, unexpectedly
+   buggy historical flight data).
+
+ * `commission-<something>': Flights used for testing that a
+   particular subset of the hosts are working properly.  The hosts to
+   be tested should be blessed `commission-whatever' during
+   commissioning, and that blessing removed and replaced with `real'
+   when the hosts are ready.
+
+ * `real-bisect' and `adhoc-bisect': These are found only as the
+   blessing of finished flights.  (This is achieved by passing
+   *-bisect to sg-execute-flight.)  This allows the archaeologist
+   tools to distinguish full flights from bisection steps.
+
+   The corresponding intended blessing (as found in the `intended'
+   column of the flights table) is `real'.  So the hosts used by the
+   automatic host allocator are those flagged `blessed-real' or
+   `blessed-adhoc' - there are no separate blessed-*-bisect hostflags.
+
+There are also blessings for unfinished flights:
+
+ * `constructing': The flight metadata (jobs, runvars, etc.) is still
+   being populated.  The tools for flight construction (and flight
+   construct editing) generally insist that they operate only on
+   flights in this state.
+
+ * `running': At least one of the jobs in this flight has started.
+   Flight execution software will generally (perhaps via standard
+   library calls) mark a flight `running' if they find it
+   `constructing', and refuse to operate on other flights.
+
+ * `broken': Something went catastrophically wrong.
+
+ * `broken-*': Conventionally set by hand when a flight needs to be
+   `un-blessed', perhaps because it contains data misleading to the
+   archaeologists.
+
+   A flight's blessings can be manually changed with cs-flight-bless:
+      ./cs-flight-bless FLIGHT broken-real real
+   updates FLIGHT to be marked `broken' rather than `real'.
+   (cs-flight-bless matches the existing blessing against its final
+   argument, to avoid accidents.)
+
+Note that osstest is generally `crash-only software': nothing will
+`clean up' a crashed flight and set its blessing to `broken'.  Flights
+which were in progress once upon a time but never completed, because
+they crashed, are simply left with whatever blessing they had at the
+time.
+
+There is a special exception to the tools' flight status checks: any
+flight whose blessing contains `play' can be operated on out of order.
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel