[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Wg-test-framework] [Minutes] December Test WG framework meeting

== Attendees ==

* Lars Kurth (chair)

* Aravind Gopalakrishnan (AMD)

* Greg Lutostanski (Calxeda)

* Chris Shepherd (Citrix)

* Donald D Dugger (Intel)

* Don Slutz (Verizon)


== Costing of Test Framework (outside of the LF) ==

We originally assumed that the Linux Foundation could host a test framework for the Xen Project : however due to the heterogeneity of the environment this is not feasible. The Linux Foundation does not have the capability to support the Xen Project need’s.


Chris Shepherd put together a costing based on

a)      External COLO space

b)      Procuring a set of machines

c)       Initial setup (one-off-cost)

d)      As well as ongoing HW maintenance together with effort needed for remote management


We discussed this proposal at the meeting and came to the conclusion that the costing is reasonable (assuming that the community wants a Test Framework)


{{Action|Lars}} Incorporate into budget {{Tick}}


The open question was how to handle resourcing for HW maintenance and Remote Management in principle. There are several options:

* Vendor(s) to donate time and effort

* Project to fund a head (working at the Linux Foundation, this looks unlikely given the previous conversation with the Linux Foundation)

* Funding resource from within the Xen Project, but have resource work for a vendor


The consensus was, that the Advisory Board needs to make a recommendation on what is acceptable.


{{Action|Lars}} to raise at next Advisory Board meeting


On hosting the Hardware on a vendor site , there were a number of options besides hosting at a 3rd party:

* Don from Intel had discussed this internally and hosting machines at Intel may be a possibility

* Chris from Citrix, clarified that Citrix could do this too in principle (initially we assumed otherwise)

* There is also a possibility that Verizon could do this also


{{Action|DonD, Chris, DonS) to follow up internally


But again, this would need to be discussed at the Advisory Board.


We also had a discussion on what type of hardware could be hosted: the constraint is that *only* publicly available Hardware can be included. Including prototype Hardware is not practical due to NDAs and other agreements that would be required.


We also had a brief discussion on whether there would be value for Hardware Vendors to set up a test infrastructure locally for *unusual* or *prototype* hardware. The consensus was that this may be of interest. Intel felt, that this was only valuable for *regression testing*, not for *development*.


== Update on XenRT (Chris) ==

No progress due to:

* http://osuosl.org/ has been unresponsive

* Chris stated, that he has not been pushing hard enough


{{Action|Chris}} Get this going again


== Update on xen-devel discussion on testing for Xen ==


Lars summarized the thread called “Testing for the Xen Project” on xen devel mailing lists.


The agreement by the community seems to be:

* We have some local test suites which could be run, but are probably not as they are poorly documented

* We expect people to do some targeted local testing (presumably  performed in a manual manner) of the features they developed and of  those which may be impacted. But we don't actually always know whether they do (although often we do spot this).

* osstest (or system testing in general) is *extremely* valuable

* building out the infrastructure for system testing (aka number and diversity of boxes) would be *extremely* valuable - this really means  funding hardware, hosting and sysadmin time

* It might also be worth considering spending some money kickstarting development of the actual tests (i.e. fleshing out the suites)

* *Test on demand* would be a nice long term goal


DonS: Don Slutz from Verizon agreed that being able to test before submissions would be extremely valuable


* Some members of the community are intending to make OSSTest more  accessible by improving docs and sharing their experience

* At a minimum it ought to be possible to allow access to any employee of a project member, since we have the opportunity through the membership, process to put whatever paperwork and agreements (acceptable use etc) in place.


My take-away from this discussion is that we are going in the right direction. The osstest vs. XenRT discussion does however remain sensitive. To move to XenRT, a number of doubts by the community would need to be addressed.


== AOB ==

* Next meeting: Lars is travelling for the January meeting and asked whether anybody else would volunteer the next meeting. Nobody stepped up, so we will have to cancel the January meeting


Wg-test-framework mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.