Re: [Xen-devel] [OSSTEST PATCH v3 0/3] Test case for cpupools

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Sat, 2015-10-03 at 02:39 +0200, Dario Faggioli wrote:
> There are some host related considerations. In fact, this test case
> requires
> that an host with at least 2 pCPUs is used. v2 was failing the test, if that
> was not the case. Now, I'm just skipping doing pretty much everything, but I'm
> not reporting failure.  In any case, while reviewing v2, IanC said:
> 
>  "The proper fix would be a property in the hostdb which was used to
>   constrain which hosts the jobs containing this test could run on. (e.g.
>   we have pcipassthrough-nic).
> 
>   Maybe this way is OK until we find we are commissioning a machine with a
>   single CPU, at which point this failure will seem pretty obvious. Ian?"
> 
> I do like this. I, actually, even started to do this already, for other
> reasons, and I am fine continuing working to make this happen, but I'd need
> some help, or at least some pointers.
> 
> I've never interacted with the hostdb, so I'd appreciate pointers for knowing
> where to look.

It's basically a (host,key)=>(value) database table, in standalone mode it
is HostProp_<host>_<key> in your cfg (as you've found) and in Executive
mode it is an actual database table.

From the PoV of code though you just use get_host_property and don't care
which it comes from.

> What I drafted was right that kind (or so it looks to me) of host properties,
> but meant at standalone mode, and I did it like in the attached patch... Any
> thoughts on this?

Personally I don't think this "look in the db and fallback to asking the
host" behaviour is desirable. I think this information should always be in
the host db.

That's not to say you couldn't also have a helper script which queries the
hosts and provides the output in a form which is easy to massage into the
db. That could even be in the form of a specialised setup flight with jobs
which would install the host and query it to produce (or even update) the
necessary db contents (probably not live updates, just in a convenient form
to put into the db).

> What I also miss, in both the cases of standalone mode config file defined
> properties, and hostdb ones, it the logic for telling the host allocator to
> consider such properties when choosing the host to be used for a job. I'll go
> studying how to do it, but if anyone feels the irresistible need of advising,
> feel free to go ahead... :-)

Until recently the Executive allocator could only make decisions based on
host "flags", which is a separate table of boolean host properties. The
host allocator matches the all_hostflags+<ident>_hostflags (where <ident>
is "host" or "src_host" etc.

e.g. in a typical flight:

    test-armhf-armhf-xl-xsm all_hostflags  
arch-armhf,arch-xen-armhf,suite-jessie,purpose-test                             
                   
    build-amd64             host_hostflags 
share-build-jessie-amd64,arch-amd64,suite-jessie,purpose-build                  
       

So for build-amd64 the chosen host has to have those flags. For the test-*
one any hosts involved in the test have to have those flags. It's not test
-* vs build-* which determines which you use, just a coincidence here. If
e.g. a migration test had differing requirments for source and host then
src_host_hostflags might differ from dst_host_hostflags and all_hostflags
would be common.

Until recently all you could do was simple matching of those boolean
properties (there are actually some magic ones, like equiv-1, please ignore
those).

Recently though I added support for evaluating hosts based on their host
props as well as flags. See 9f33933526d3 "Add support for selecting
resources based on their properties." which added one particular comparator
in Osstest::ResourceCondition::PropMinVer as a way to limit things to hosts
which were supported by a given version of Linux (e.g. requisite drivers
and such). 

So for example in a linux-4.1 flight we now have:

    test-armhf-armhf-xl-vhd all_hostflags 
arch-armhf,arch-xen-armhf,suite-jessie,purpose-test,PropMinVer:LinuxKernelMin:4.1
                    

Where the last bit causes the hosts LinuxKernelMin property to be compared
against 4.1. Any host which has a LinuxKernelMin host prop won't run linux
-4.1 tests. In this case absence of the prop means "no specific
requirement", but other comparators may differ.

It would be quite easy to add a PropMin module which similarly required
that some host prop existed and had a particular minimum value. Then you
could use "PropMin:Cpus:2" to ensure you get a host with a Cpus hostprop of
at least 2.

The is no allocator in Standalone mode, you have to provide a host which
meets the requirements yourself and if you don't it might break.

Clear as mud?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel