From wg-test-framework-bounces@lists.xenproject.org Thu Feb 06 15:24:20 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Thu, 06 Feb 2014 15:24:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBQoO-0003vH-Pa for archives@lists.xenproject.org; Thu, 06 Feb 2014 15:24:20 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBQoN-0003vC-8i for wg-test-framework@lists.xenproject.org; Thu, 06 Feb 2014 15:24:19 +0000 Received: from [193.109.254.147:18673] by server-7.bemta-14.messagelabs.com id 2C/C2-23424-229A3F25; Thu, 06 Feb 2014 15:24:18 +0000 X-Env-Sender: lars.kurth@citrix.com X-Msg-Ref: server-7.tower-27.messagelabs.com!1391700256!2525594!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 14906 invoked from network); 6 Feb 2014 15:24:17 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-7.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 6 Feb 2014 15:24:17 -0000 X-IronPort-AV: E=Sophos;i="4.95,793,1384300800"; d="scan'208";a="98612915" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 06 Feb 2014 15:24:15 +0000 Received: from AMSPEX01CL03.citrite.net (10.69.46.34) by FTLPEX01CL02.citrite.net (10.13.107.79) with Microsoft SMTP Server (TLS) id 14.2.342.4; Thu, 6 Feb 2014 10:24:15 -0500 Received: from AMSPEX01CL01.citrite.net ([169.254.6.176]) by AMSPEX01CL03.citrite.net ([10.69.46.34]) with mapi id 14.02.0342.004; Thu, 6 Feb 2014 16:24:14 +0100 From: Lars Kurth To: "wg-test-framework@lists.xenproject.org" Thread-Topic: Vote results: Community Representative, Face-2-face meetings Thread-Index: Ac8jThL9zMF8saP7Ri+UuldmINEO1A== Date: Thu, 6 Feb 2014 15:24:13 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.2.160] MIME-Version: 1.0 X-DLP: MIA2 Cc: Doug Clark Subject: [Wg-test-framework] Vote results: Community Representative, Face-2-face meetings X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org Hi everyone, > *[Vote1]* I would like to invite a community representative to join > the WG. The process is to ask for a volunteer on the mailing lists and > for that volunteer to be confirmed by a community vote. The rep would > have an equal vote to all of you. 3 votes in favour, no objections. The vote carries. I will send out a mail to the xen-devel list in a moment to ask for a community representative on the WG to self-nominate. Hopefully he/she can be confirmed before the next WG meeting and attend. > *[Vote2]* Linux Foundation Collab Summit is from March 26-28 in NAPA. > This is an opportunity for a face-2-face meeting. Please vote +1, if rep to step up > you are planning to attend Not enough traction for a face-2-face meeting in March. So let's not do it. Let me introduce Doug Clark, who is Staff SW Engineer in the Citrix Test team. Doug works for Chris Shepherd and may take over as Citrix representative on the Test WG at some point. Doug, you can find the list archive at http://lists.xenproject.org/archives/html/wg-test-framework/ I will be adding Doug to the WG mailing list and he will participate at the next WG meeting. I will propose the agenda for the next WG meeting later today. Best Regards Lars _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework From wg-test-framework-bounces@lists.xenproject.org Thu Feb 06 15:24:20 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Thu, 06 Feb 2014 15:24:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBQoO-0003vH-Pa for archives@lists.xenproject.org; Thu, 06 Feb 2014 15:24:20 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBQoN-0003vC-8i for wg-test-framework@lists.xenproject.org; Thu, 06 Feb 2014 15:24:19 +0000 Received: from [193.109.254.147:18673] by server-7.bemta-14.messagelabs.com id 2C/C2-23424-229A3F25; Thu, 06 Feb 2014 15:24:18 +0000 X-Env-Sender: lars.kurth@citrix.com X-Msg-Ref: server-7.tower-27.messagelabs.com!1391700256!2525594!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 14906 invoked from network); 6 Feb 2014 15:24:17 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-7.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 6 Feb 2014 15:24:17 -0000 X-IronPort-AV: E=Sophos;i="4.95,793,1384300800"; d="scan'208";a="98612915" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 06 Feb 2014 15:24:15 +0000 Received: from AMSPEX01CL03.citrite.net (10.69.46.34) by FTLPEX01CL02.citrite.net (10.13.107.79) with Microsoft SMTP Server (TLS) id 14.2.342.4; Thu, 6 Feb 2014 10:24:15 -0500 Received: from AMSPEX01CL01.citrite.net ([169.254.6.176]) by AMSPEX01CL03.citrite.net ([10.69.46.34]) with mapi id 14.02.0342.004; Thu, 6 Feb 2014 16:24:14 +0100 From: Lars Kurth To: "wg-test-framework@lists.xenproject.org" Thread-Topic: Vote results: Community Representative, Face-2-face meetings Thread-Index: Ac8jThL9zMF8saP7Ri+UuldmINEO1A== Date: Thu, 6 Feb 2014 15:24:13 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.2.160] MIME-Version: 1.0 X-DLP: MIA2 Cc: Doug Clark Subject: [Wg-test-framework] Vote results: Community Representative, Face-2-face meetings X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org Hi everyone, > *[Vote1]* I would like to invite a community representative to join > the WG. The process is to ask for a volunteer on the mailing lists and > for that volunteer to be confirmed by a community vote. The rep would > have an equal vote to all of you. 3 votes in favour, no objections. The vote carries. I will send out a mail to the xen-devel list in a moment to ask for a community representative on the WG to self-nominate. Hopefully he/she can be confirmed before the next WG meeting and attend. > *[Vote2]* Linux Foundation Collab Summit is from March 26-28 in NAPA. > This is an opportunity for a face-2-face meeting. Please vote +1, if rep to step up > you are planning to attend Not enough traction for a face-2-face meeting in March. So let's not do it. Let me introduce Doug Clark, who is Staff SW Engineer in the Citrix Test team. Doug works for Chris Shepherd and may take over as Citrix representative on the Test WG at some point. Doug, you can find the list archive at http://lists.xenproject.org/archives/html/wg-test-framework/ I will be adding Doug to the WG mailing list and he will participate at the next WG meeting. I will propose the agenda for the next WG meeting later today. Best Regards Lars _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework From wg-test-framework-bounces@lists.xenproject.org Fri Feb 07 16:55:38 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Fri, 07 Feb 2014 16:55:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBoiI-0002yH-3y for archives@lists.xenproject.org; Fri, 07 Feb 2014 16:55:38 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBoiH-0002y7-7q for wg-test-framework@lists.xenproject.org; Fri, 07 Feb 2014 16:55:37 +0000 Received: from [193.109.254.147:38526] by server-3.bemta-14.messagelabs.com id F9/3C-00432-80015F25; Fri, 07 Feb 2014 16:55:36 +0000 X-Env-Sender: Ian.Jackson@citrix.com X-Msg-Ref: server-8.tower-27.messagelabs.com!1391792134!2821529!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 30314 invoked from network); 7 Feb 2014 16:55:35 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-8.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 7 Feb 2014 16:55:35 -0000 X-IronPort-AV: E=Sophos;i="4.95,801,1384300800"; d="scan'208";a="99027909" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 07 Feb 2014 16:55:33 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Fri, 7 Feb 2014 11:55:33 -0500 Received: from mariner.cam.xci-test.com ([10.80.2.22] helo=mariner.uk.xensource.com) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WBoiC-00049H-Sf; Fri, 07 Feb 2014 16:55:32 +0000 Received: from iwj by mariner.uk.xensource.com with local (Exim 4.80) (envelope-from ) id 1WBoiC-00017p-IF; Fri, 07 Feb 2014 16:55:32 +0000 From: Ian Jackson MIME-Version: 1.0 Message-ID: <21237.4100.356986.496721@mariner.uk.xensource.com> Date: Fri, 7 Feb 2014 16:55:32 +0000 To: Newsgroups: chiark.mail.xen.devel In-Reply-To: <52F3B0C6.9050904@xen.org> References: <52F3B0C6.9050904@xen.org> X-Mailer: VM 8.1.0 under 23.4.1 (i486-pc-linux-gnu) X-DLP: MIA1 Cc: wg-test-framework@lists.xenproject.org, "xen-devel@lists.xen.org" Subject: Re: [Wg-test-framework] [Xen-devel] Looking for a volunteer to represent the Xen Project developer community at Test Working group meetings X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org Lars Kurth writes ("[Xen-devel] Looking for a volunteer to represent the Xen Project developer community at Test Working group meetings"): > at the last WG group meeting I had an action to call a vote on whether > we should open up the Test Framework WG to a community member. We had > the vote and it was carried. Thanks. I'd like to volunteer for this role. > == What type of person are we looking for == > 1) Can represent the Xen Project Developer community : this means you > need to be an active member of the community (e.g. a maintainer or > committer) I'm a tools maintainer, and committer. > 2) Has some understanding of how testing in the Xen Community works > today and how it should work. > 3) Ideally you also have written tests for Xen before. I mostly wrote run the existing test system. > == Responsibilities and Rights == > 1) You are presenting the views of the community, not those of your > employee or your own. Right. I'm open to questions on how I would approach this task. > 2) This means that you will need to consult with the developer community > and understand the key issues on testing. Indeed. > 3) You commit to attending WG meetings and monitor and participate on > wg-test-framework@lists.xenproject.org I can do that. Thanks, Ian. _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework From wg-test-framework-bounces@lists.xenproject.org Fri Feb 07 16:55:38 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Fri, 07 Feb 2014 16:55:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBoiI-0002yH-3y for archives@lists.xenproject.org; Fri, 07 Feb 2014 16:55:38 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBoiH-0002y7-7q for wg-test-framework@lists.xenproject.org; Fri, 07 Feb 2014 16:55:37 +0000 Received: from [193.109.254.147:38526] by server-3.bemta-14.messagelabs.com id F9/3C-00432-80015F25; Fri, 07 Feb 2014 16:55:36 +0000 X-Env-Sender: Ian.Jackson@citrix.com X-Msg-Ref: server-8.tower-27.messagelabs.com!1391792134!2821529!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 30314 invoked from network); 7 Feb 2014 16:55:35 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-8.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 7 Feb 2014 16:55:35 -0000 X-IronPort-AV: E=Sophos;i="4.95,801,1384300800"; d="scan'208";a="99027909" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 07 Feb 2014 16:55:33 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Fri, 7 Feb 2014 11:55:33 -0500 Received: from mariner.cam.xci-test.com ([10.80.2.22] helo=mariner.uk.xensource.com) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WBoiC-00049H-Sf; Fri, 07 Feb 2014 16:55:32 +0000 Received: from iwj by mariner.uk.xensource.com with local (Exim 4.80) (envelope-from ) id 1WBoiC-00017p-IF; Fri, 07 Feb 2014 16:55:32 +0000 From: Ian Jackson MIME-Version: 1.0 Message-ID: <21237.4100.356986.496721@mariner.uk.xensource.com> Date: Fri, 7 Feb 2014 16:55:32 +0000 To: Newsgroups: chiark.mail.xen.devel In-Reply-To: <52F3B0C6.9050904@xen.org> References: <52F3B0C6.9050904@xen.org> X-Mailer: VM 8.1.0 under 23.4.1 (i486-pc-linux-gnu) X-DLP: MIA1 Cc: wg-test-framework@lists.xenproject.org, "xen-devel@lists.xen.org" Subject: Re: [Wg-test-framework] [Xen-devel] Looking for a volunteer to represent the Xen Project developer community at Test Working group meetings X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org Lars Kurth writes ("[Xen-devel] Looking for a volunteer to represent the Xen Project developer community at Test Working group meetings"): > at the last WG group meeting I had an action to call a vote on whether > we should open up the Test Framework WG to a community member. We had > the vote and it was carried. Thanks. I'd like to volunteer for this role. > == What type of person are we looking for == > 1) Can represent the Xen Project Developer community : this means you > need to be an active member of the community (e.g. a maintainer or > committer) I'm a tools maintainer, and committer. > 2) Has some understanding of how testing in the Xen Community works > today and how it should work. > 3) Ideally you also have written tests for Xen before. I mostly wrote run the existing test system. > == Responsibilities and Rights == > 1) You are presenting the views of the community, not those of your > employee or your own. Right. I'm open to questions on how I would approach this task. > 2) This means that you will need to consult with the developer community > and understand the key issues on testing. Indeed. > 3) You commit to attending WG meetings and monitor and participate on > wg-test-framework@lists.xenproject.org I can do that. Thanks, Ian. _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework From wg-test-framework-bounces@lists.xenproject.org Fri Feb 07 18:27:09 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Fri, 07 Feb 2014 18:27:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBq8r-0000GZ-Sb for archives@lists.xenproject.org; Fri, 07 Feb 2014 18:27:09 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBq8q-0000EG-3H; Fri, 07 Feb 2014 18:27:08 +0000 Received: from [85.158.143.35:55171] by server-3.bemta-4.messagelabs.com id AE/F0-11539-B7525F25; Fri, 07 Feb 2014 18:27:07 +0000 X-Env-Sender: lars.kurth@citrix.com X-Msg-Ref: server-2.tower-21.messagelabs.com!1391797624!4011650!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 26929 invoked from network); 7 Feb 2014 18:27:05 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-2.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 7 Feb 2014 18:27:05 -0000 X-IronPort-AV: E=Sophos;i="4.95,802,1384300800"; d="scan'208,217";a="99056175" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 07 Feb 2014 18:27:03 +0000 Received: from AMSPEX01CL02.citrite.net (10.69.46.33) by FTLPEX01CL01.citrite.net (10.13.107.78) with Microsoft SMTP Server (TLS) id 14.2.342.4; Fri, 7 Feb 2014 13:27:02 -0500 Received: from AMSPEX01CL01.citrite.net ([169.254.6.176]) by AMSPEX01CL02.citrite.net ([10.69.46.33]) with mapi id 14.02.0342.004; Fri, 7 Feb 2014 19:27:01 +0100 From: Lars Kurth To: "wg-test-framework@lists.xenproject.org" Thread-Topic: Proposed Agenda for next Test Framework WG meeting Thread-Index: Ac8kJ2Rvzh4fHDKeT7SvzOeKEf0ihA== Date: Fri, 7 Feb 2014 18:27:00 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.68.19.222] MIME-Version: 1.0 X-DLP: MIA1 Cc: "advisory-board@lists.xenproject.org" Subject: [Wg-test-framework] Proposed Agenda for next Test Framework WG meeting X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1338677578587307231==" Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org --===============1338677578587307231== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_" --_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi all, please find attached the agenda for the next Test Framework working group m= eeting. The meeting is on Feb 13th at 14:00 UTC. Please check whether you = have the invite. The current list of WG members can be found here : http://wiki.xenproject.o= rg/wiki/AB_WG/Test_Framework If your company is on the Advisory Board, and= you would like to influence the direction of the Test Framework WG, you ca= n still nominate a representative for the Test Framework WG. Please drop me= a line, if you want to join. The good news is that the Advisory Board has approved the budget that we as= ked for (which is outlined below). The total allocated budget for the Test = Framework is USD 190,612.50 (including Linux Foundation charges) - this is = a conservative estimate. We may be able to build a test system with less th= at the allocated budget or build a larger system. Note that Konrad Wilk from Oracle cannot attend the next few WG meetings, a= nd has not found a rep from Oracle to cover for him. I agreed with Konrad, = that we will try and use the mailing lists for decisions, reviewing documen= ts, etc. as much as possible and use the meetings to raise concerns and iss= ues. I also added Ian Jackson to this email thread and the mailing list, as= he is maintaining and running OSSTest. OSSTest is the existing test framew= ork for the Xen Project which is hosted by Citrix and his input is thus imp= ortant for this group. Ian also nominated himself as community rep for the = WG : we do have to follow the process that is outlined in the e-mail titled= "Looking for a volunteer to represent the Xen Project developer community = at Test Working". If there are no other nominees and no objections by the m= aintainer community, Ian would be confirmed as community representative bef= ore the WG meeting. =3D=3D Agenda =3D=3D I would like to review the Strawman of a plan at the meeting (see below). F= eedback by e-mail prior to the meeting is desired. If I get feedback before= the Test Framework WG meeting I will try to incorporate. At a minimum, I would like to * Get agreement on some of the basic principles in the Strawman ** The split between short term and long term goals ** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to prov= ide the information that we need in the short and long term ** Identify a technical leader : in the best case a WG member volunteers an= d will be confirmed by the group. *** If nobody steps up, we would set a deadline. *** If the deadline passes and still nobody steps up, I will need to ask th= e Advisory Board for advice on how to proceed and whether a shared Test Inf= rastructure is still viable ** Identify a deputy who will work alongside me on non-technical questions * As a stretch goal, we may be able to approve sections of the plan, or the= entire plan =3D=3D Strawman of a Plan to move forward =3D=3D =3D=3D=3D Ownership =3D=3D=3D A) We need someone from this group with technical expertise and insight int= o testing Xen who can lead and drive technical tasks in the WG. I cannot fu= lfil this role and without it, I do not believe we will get anywhere. I am = looking for one or several of the vendors on this group to step up. Given t= hat both OSSTest and XenRT are currently the only viable solutions for the = test framework on the table, the reality is that Citrix needs to step up. U= nless we throw another solution into the mix. The tasks at hand that the technical leader will need to drive are : * Describe/specify a viable Hardware set-up that we can afford : my suggestion here would be that as a starting point, Ian Jackson and Ch= ris Shepherd work together and work out a minimum environment that works fo= r both OSSTest and XenRT. This would allow us to de-couple the "what are we doing in the long term= " from all the logistical questions. If necessary (and we have enough budge= t), we can extend the setup as needed * Establish concrete requirements for a COLO provider (ideally with a short= -list of possible vendors) * Establish requirements for sysadmin and support * Establish a plan that lists the main tasks that are needed to get us to a= system that is up and running in the short term (I can help with the plan) * Work with all stake-holders on a long-term plan (I can help with the plan= ) B) Ownership of non-technical areas of the WG. I can do this, but would pre= fer if someone else within the group would act as a deputy and work with me= . I travel quite frequently and thus work items may get stuck. The tasks at hand that this WG member will need to drive are : * Work with the community representative and the developer community to ens= ure that what we propose is valuable to the Xen project developers and woul= d in fact be used by them * Handle any contractual issues (e.g. finding and getting a COLO, buying li= censes, buying machines) - in other words act as the interface to the Linux= Foundation * Resolving any issues related to sysadmin and support (e.g. hiring a contr= actor or part-time resource as needed and resolve any practical issues arou= nd it) * Work on proposals for process related questions, such as ** Who can access, how is access managed ** How do we add new HW in future, how do we add extra software configurati= ons ** Etc. =3D=3D=3D Short Term Goal =3D=3D=3D I think we should transition the existing OSSTest system to new hardware im= mediately and then consider how to evolve our test system in the future. We= want to end up with a solution that the community can use to test xen.git,= right away, at our current level of service or better. At the moment only= OSSTest can do this : there are too many open questions related to XenRT a= nd progress on a XenRT proof of concept has been very slow (this has pretty= much stalled since August 2013). However we should choose the infrastructu= re such that both OSSTest and XenRT can run on it. This approach has several advantages * It decouples creating the infrastructure from the choice of test system * It allows us to iron out any logistical and practical issues that owning = an independently hosted test infrastructure involves * It delivers immediate value to the community and shows progress with litt= le disruption. If we stay in limbo, the Xen Project developer community wil= l not write any new tests. Developers will simply wait for a decision. * It leaves all options open, while creating a motivation to work on the L= ong Term Goals and ensure we end up with a solution that works. If we fail = to agree and do not deliver on the Long Term Goals, the Xen Project will st= ill have an independent test infrastructure. However the infrastructure may= be less aligned with the commercial interests of the Advisory Board. In ot= her words, we create a motivation to resolve the long term goals more quick= ly than otherwise would exist. =3D=3D=3D Long Term Goal =3D=3D=3D To ensure that we do the right thing in the long run, I believe we need to = take stock of the technical options we have: right now we have a choice of = OSSTest and XenRT. There may be others as well, which can of course be prop= osed. To figure out what is best in the long run, we need to get a concrete under= standing of * The level of testing that can be achieved with the solutions we have avai= lable today (maybe in form of a very coarse list such as "boot a variety of= different guests", ... for per solution) * The Hardware requirements that are needed to do this * Software requirements (e.g. licenses) * Highlight any issues, work that needs to be performed, etc. BEFORE the so= lution can be deployed * Highlight any concerns that may prevent the Xen Project developer communi= ty from adopting each solution and mitigations on how we may address these (I can help with this and will work with the community representative) * And there may be other areas that I am missing To drive this forward, we need a champion for each proposal who will work w= ith the WG (ideally is a member of it) and who commits to providing the nec= essary information and if necessary secures resources in their respective o= rganization to ensure that gaps and community concerns can be addressed. It= also requires input from the Community rep on the WG. Best Regards Lars ------------------------- Source: Chris Shepherd, Citrix Based on operational expertise with XenRT Item Item Cost Notes One off costs for hardware Core infrastructure (filers, etc.) 24,000.00 A pool of servers and shared storage to run the various VMs you need to run= the network, monitor infrastructure, pxe boot, dhcp, scheduling etc. Osste= st would presumably need something similar. XenRT 'site' infrastructure 12,000.00 XenRT can be built up into multiple 'sites', each consisting of a number of= servers (we have sites of 16, 32 and 64 servers). Each site needs its own = small pool of servers with shared storage and switches to run the site. For= a small XenRT deployment (one site) we could probably share the core infra= structure for the site infrastructure, thereby removing (or at least postpo= ning) this cost. 16 mid range test servers at $4K each 64,000.00 Would only include off-the shelf hardware (no prototypes) Could be lower spec machines (OSSTest machines are at $1K) A reasonable approach would be to use a mixture of machines up to $4K, whic= h should get us to more than 16 servers Total: 100,000.00 Colo cost / month Cost per rack / month 2,500.00 Inclusive of space, power and cooling 20Mbps internet with high SLA / month 1,000.00 Could reduce this cost very significantly in return for lower SLA Sysadmin for HW only 500.00 Based on experience with XenRT, we will probably need 1 day per month for r= outine maintenance, plus time to setup any new equipment. Don't have a mechanism to do this (avg hourly rate $30/h) Conservatively costing this at $500 Total/month: 4,000.00 Remote admin and support of environment 20% of a tools dev /devops type person This is based on Citrix effort to support Citrix' XenRT lab which consists = of well over 200 servers at 2 separate physical sites (US west coast and UK= ). Includes: a) General user support e.g. answering usage queries including helping trou= ble-shoot failed tests (that may be due to infrastructure problems rather t= han product test) b) Clearing out files that are taking up disk space c) Monitoring server & network health and fixing broken machines e.g. serve= r lost BIOS setting, network connectivity etc I don't know the typical cost for such a profile and of course there is the= issue that we cannot employ 20% of a person. An average test engineer sala= ry in the US costs $88000 per annum. It seems that the overhead for OSSTest is higher (Ian Jackson says he spend= s 50% of his time looking after fewer machines) Estimated Total/month: 3,750.00 Conservatively assuming 50% of an engineer for $90K (unloaded) as a ball-pa= rk figure for feasibility purpuses only. This item would need to be revised= after some more discussion within the Advisory Board. --_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi all,

 

please find attached the agenda for the next Test Fr= amework working group meeting.  The meeting is on Feb 13th = at 14:00 UTC. Please check whether you have the invite.

 

The current list of WG members can be found here : <= a href=3D"http://wiki.xenproject.org/wiki/AB_WG/Test_Framework"> http://wiki.xenproject.org/wiki/AB_WG/Test_Framework  If your comp= any is on the Advisory Board, and you would like to influence the direction= of the Test Framework WG, you can still nominate a representative for the = Test Framework WG. Please drop me a line, if you want to join.

 

The good news is that the Advisory Board has approve= d the budget that we asked for (which is outlined below). The total allocat= ed budget for the Test Framework is USD 190,612.50 (including Linux Foundat= ion charges) – this is a conservative estimate. We may be able to build a test system with less that the allocat= ed budget or build a larger system.

 

Note that Konrad Wilk from Oracle cannot attend the = next few WG meetings, and has not found a rep from Oracle to cover for him.= I agreed with Konrad, that we will try and use the mailing lists for decis= ions, reviewing documents, etc. as much as possible and use the meetings to raise concerns and issues. I also= added Ian Jackson to this email thread and the mailing list, as he is main= taining and running OSSTest. OSSTest is the existing test framework for the= Xen Project which is hosted by Citrix and his input is thus important for this group. Ian also nominated = himself as community rep for the WG : we do have to follow the process that= is outlined in the e-mail titled “Looking for a volunteer to represe= nt the Xen Project developer community at Test Working”. If there are no other nominees and no objections b= y the maintainer community, Ian would be confirmed as community representat= ive before the WG meeting.

 

=3D=3D Agenda =3D=3D

 

I would like to review the Strawman of a plan at the= meeting (see below). Feedback by e-mail prior to the meeting is desired. I= f I get feedback before the Test Framework WG meeting I will try to incorpo= rate.

 

At a minimum, I would like to

* Get agreement on some of the basic principles in t= he Strawman

** The split between short term and long term goals<= o:p>

** Agreement by stake-holders, i.e. Ian Jackson and = Chris Shepherd, to provide the information that we need in the short and lo= ng term

** Identify a technical leader : in the best case a = WG member volunteers and will be confirmed by the group.
*** If nobody steps up, we would set a deadline.

*** If the deadline passes and still nobody steps up= , I will need to ask the Advisory Board for advice on how to proceed and wh= ether a shared Test Infrastructure is still viable

** Identify a deputy who will work alongside me on n= on-technical questions

* As a stretch goal, we may be able to approve secti= ons of the plan, or the entire plan

 

=3D=3D Strawman of a Plan to move forward =3D=3D

 

=3D=3D=3D Ownership =3D=3D=3D

 

A) We need someone from this group with technical ex= pertise and insight into testing Xen who can lead and drive technical tasks= in the WG. I cannot fulfil this role and without it, I do not believe we w= ill get anywhere. I am looking for one or several of the vendors on this group to step up. Given that both OS= STest and XenRT are currently the only viable solutions for the test framew= ork on the table, the reality is that Citrix needs to step up. Unless we th= row another solution into the mix.

 

The tasks at hand that the technical leader will nee= d to drive are :

* Describe/specify a viable Hardware set-up that we = can afford :
   my suggestion here would be that as a starting point, Ian Jack= son and Chris Shepherd work together and work out a minimum environment tha= t works for both OSSTest and XenRT.
   This would allow us to de-couple the “what are we d= oing in the long term” from all the logistical questions. If necessar= y (and we have enough budget), we can extend the setup as needed=

* Establish concrete requirements for a COLO provide= r (ideally with a short-list of possible vendors)

* Establish requirements for sysadmin and support

* Establish a plan th= at lists the main tasks that are needed to get us to a system that is up an= d running in the short term (I can help with the plan)
* Work with all stake-holders on a long-term plan (I can help with the plan= )

B) Ownership of non-technical areas of the WG. I can= do this, but would prefer if someone else within the group would act as a = deputy and work with me. I travel quite frequently and thus work items may = get stuck.

 

The tasks at hand that this WG member will need to d= rive are :

* Work with the community representative and the dev= eloper community to ensure that what we propose is valuable to the Xen proj= ect developers and would in fact be used by them

* Handle any contractual issues (e.g. finding and ge= tting a COLO, buying licenses, buying machines) – in other words act = as the interface to the Linux Foundation

* Resolving any issues related to sysadmin and suppo= rt (e.g. hiring a contractor or part-time resource as needed and resolve an= y practical issues around it)

* Work on proposals for process related questions, s= uch as

** Who can access, how is access managed<= /p>

** How do we add new HW in future, how do we add ext= ra software configurations

** Etc.

=3D=3D=3D Short Term Goal =3D=3D=3D

 

I think we should transition the existing OSSTest= system to new hardware immediately and then consider how to evolve our tes= t system in the future. We want to end up with a solution that the communit= y can use to test xen.git, right away, at our current level of service or better.  At the moment only OSSTes= t can do this : there are too many open questions related to XenRT and prog= ress on a XenRT proof of concept has been very slow (this has pretty much s= talled since August 2013). However we should choose the infrastructure such that both OSSTest and XenRT can run = on it.

 

This approach has several advantages

* It decouples creating the infrastructure from t= he choice of test system

* It allows us to iron out any logistical and pra= ctical issues that owning an independently hosted test infrastructure invol= ves

* It delivers immediate value to the community an= d shows progress with little disruption. If we stay in limbo, the Xen Proje= ct developer community will not write any new tests. Developers will simply= wait for a decision.

* It leaves all options open,  while creatin= g a motivation to work on the Long Term Goals and ensure we end up with a s= olution that works. If we fail to agree and do not deliver on the Long Term= Goals, the Xen Project will still have an independent test infrastructure. However the infrastructure may be less= aligned with the commercial interests of the Advisory Board. In other word= s, we create a motivation to resolve the long term goals more quickly than = otherwise would exist.

 

=3D=3D=3D Long Term Goal =3D=3D=3D

 

To ensure that we do = the right thing in the long run, I believe we need to take stock of the tec= hnical options we have: right now we have a choice of OSSTest and XenRT. Th= ere may be others as well, which can of course be proposed.

To figure out what is= best in the long run, we need to get a concrete understanding of

* The level of testing that can be achieved with the= solutions we have available today (maybe in form of a very coarse list suc= h as “boot a variety of different guests”, … for per solu= tion)

* The Hardware requirements that are needed to do th= is

* Software requirements (e.g. licenses)

* Highlight any issues, work that needs to be perfor= med, etc. BEFORE the solution can be deployed

* Highlight any concerns that may prevent the Xen Pr= oject developer community from adopting each solution and mitigations on ho= w we may address these
  (I can help with this and will work with the community represen= tative)
* And there may be other areas that I am missing


To drive this forward, we need a champion for each proposal who will work w= ith the WG (ideally is a member of it) and who commits to providing the nec= essary information and if necessary secures resources in their respective o= rganization to ensure that gaps and community concerns can be addressed. It also requires input from the C= ommunity rep on the WG.

Best Regards

Lars

 

-------------------------

 

Source: Chris = Shepherd, Citrix

Based on opera= tional expertise with XenRT

Item

Item

Cost

Notes

One off cos= ts for hardware

Core infrastru= cture (filers, etc.)

24,000.00

A pool of serv= ers and shared storage to run the various VMs you need to run the network, = monitor infrastructure, pxe boot, dhcp, scheduling etc. Osstest would presumably need something similar.

XenRT 'site' i= nfrastructure

12,000.00

XenRT can be b= uilt up into multiple ‘sites’, each consisting of a number of s= ervers (we have sites of 16, 32 and 64 servers). Each site needs its own small pool of servers with shared storage and switches to run the site= . For a small XenRT deployment (one site) we could probably share the core = infrastructure for the site infrastructure, thereby removing (or at least p= ostponing) this cost.

16 mid range t= est servers at $4K each

64,000.00

Would only inc= lude off-the shelf hardware (no prototypes)
Could be lower spec machines (OSSTest machines are at $1K)
A reasonable approach would be to use a mixture of machines up to $4K, whic= h should get us to more than 16 servers

Total:

100,000.0= 0=

Colo cost /= month

Cost per rack = / month

2,500.00

Inclusive of s= pace, power and cooling

20Mbps interne= t with high SLA / month

1,000.00

Could reduce t= his cost very significantly in return for lower SLA

Sysadmin for H= W only

500.00

= Based on experience with XenRT, we will probably need 1 day per month for r= outine maintenance, plus time to setup any new equipment.
Don’t have a mechanism to do this (avg hourly rate $30/h)
Conservatively costing this at $500

Total/month:

4,000.00= <= /span>

Remote admi= n and support of environment

20% of a tools= dev /devops type person

This is based = on Citrix effort to support Citrix' XenRT lab which consists of well over 2= 00 servers at 2 separate physical sites (US west coast and UK). Includes:
a) General user support e.g. answering usage queries including helping trou= ble-shoot failed tests (that may be due to infrastructure problems rather t= han product test)
b) Clearing out files that are taking up disk space
c) Monitoring server & network health and fixing broken machines e.g. s= erver lost BIOS setting, network connectivity etc

= I don't know the typical cost for such a profile and of course there is the= issue that we cannot employ 20% of a person. An average test engineer salary in the US costs $88000 per annum.

It seems that = the overhead for OSSTest is higher (Ian Jackson says he spends 50% of his t= ime looking after fewer machines)

Estimated Total/month:

3,750.00= <= /span>

= Conservatively assuming 50% of an engineer for $90K (unloaded) as a ball-pa= rk figure for feasibility purpuses only. This item would need to be revised after some more discussion within the Advisory Board.

 

 

--_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_-- --===============1338677578587307231== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --===============1338677578587307231==-- From wg-test-framework-bounces@lists.xenproject.org Fri Feb 07 18:27:09 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Fri, 07 Feb 2014 18:27:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBq8r-0000GZ-Sb for archives@lists.xenproject.org; Fri, 07 Feb 2014 18:27:09 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WBq8q-0000EG-3H; Fri, 07 Feb 2014 18:27:08 +0000 Received: from [85.158.143.35:55171] by server-3.bemta-4.messagelabs.com id AE/F0-11539-B7525F25; Fri, 07 Feb 2014 18:27:07 +0000 X-Env-Sender: lars.kurth@citrix.com X-Msg-Ref: server-2.tower-21.messagelabs.com!1391797624!4011650!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 26929 invoked from network); 7 Feb 2014 18:27:05 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-2.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 7 Feb 2014 18:27:05 -0000 X-IronPort-AV: E=Sophos;i="4.95,802,1384300800"; d="scan'208,217";a="99056175" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 07 Feb 2014 18:27:03 +0000 Received: from AMSPEX01CL02.citrite.net (10.69.46.33) by FTLPEX01CL01.citrite.net (10.13.107.78) with Microsoft SMTP Server (TLS) id 14.2.342.4; Fri, 7 Feb 2014 13:27:02 -0500 Received: from AMSPEX01CL01.citrite.net ([169.254.6.176]) by AMSPEX01CL02.citrite.net ([10.69.46.33]) with mapi id 14.02.0342.004; Fri, 7 Feb 2014 19:27:01 +0100 From: Lars Kurth To: "wg-test-framework@lists.xenproject.org" Thread-Topic: Proposed Agenda for next Test Framework WG meeting Thread-Index: Ac8kJ2Rvzh4fHDKeT7SvzOeKEf0ihA== Date: Fri, 7 Feb 2014 18:27:00 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.68.19.222] MIME-Version: 1.0 X-DLP: MIA1 Cc: "advisory-board@lists.xenproject.org" Subject: [Wg-test-framework] Proposed Agenda for next Test Framework WG meeting X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1338677578587307231==" Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org --===============1338677578587307231== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_" --_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi all, please find attached the agenda for the next Test Framework working group m= eeting. The meeting is on Feb 13th at 14:00 UTC. Please check whether you = have the invite. The current list of WG members can be found here : http://wiki.xenproject.o= rg/wiki/AB_WG/Test_Framework If your company is on the Advisory Board, and= you would like to influence the direction of the Test Framework WG, you ca= n still nominate a representative for the Test Framework WG. Please drop me= a line, if you want to join. The good news is that the Advisory Board has approved the budget that we as= ked for (which is outlined below). The total allocated budget for the Test = Framework is USD 190,612.50 (including Linux Foundation charges) - this is = a conservative estimate. We may be able to build a test system with less th= at the allocated budget or build a larger system. Note that Konrad Wilk from Oracle cannot attend the next few WG meetings, a= nd has not found a rep from Oracle to cover for him. I agreed with Konrad, = that we will try and use the mailing lists for decisions, reviewing documen= ts, etc. as much as possible and use the meetings to raise concerns and iss= ues. I also added Ian Jackson to this email thread and the mailing list, as= he is maintaining and running OSSTest. OSSTest is the existing test framew= ork for the Xen Project which is hosted by Citrix and his input is thus imp= ortant for this group. Ian also nominated himself as community rep for the = WG : we do have to follow the process that is outlined in the e-mail titled= "Looking for a volunteer to represent the Xen Project developer community = at Test Working". If there are no other nominees and no objections by the m= aintainer community, Ian would be confirmed as community representative bef= ore the WG meeting. =3D=3D Agenda =3D=3D I would like to review the Strawman of a plan at the meeting (see below). F= eedback by e-mail prior to the meeting is desired. If I get feedback before= the Test Framework WG meeting I will try to incorporate. At a minimum, I would like to * Get agreement on some of the basic principles in the Strawman ** The split between short term and long term goals ** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to prov= ide the information that we need in the short and long term ** Identify a technical leader : in the best case a WG member volunteers an= d will be confirmed by the group. *** If nobody steps up, we would set a deadline. *** If the deadline passes and still nobody steps up, I will need to ask th= e Advisory Board for advice on how to proceed and whether a shared Test Inf= rastructure is still viable ** Identify a deputy who will work alongside me on non-technical questions * As a stretch goal, we may be able to approve sections of the plan, or the= entire plan =3D=3D Strawman of a Plan to move forward =3D=3D =3D=3D=3D Ownership =3D=3D=3D A) We need someone from this group with technical expertise and insight int= o testing Xen who can lead and drive technical tasks in the WG. I cannot fu= lfil this role and without it, I do not believe we will get anywhere. I am = looking for one or several of the vendors on this group to step up. Given t= hat both OSSTest and XenRT are currently the only viable solutions for the = test framework on the table, the reality is that Citrix needs to step up. U= nless we throw another solution into the mix. The tasks at hand that the technical leader will need to drive are : * Describe/specify a viable Hardware set-up that we can afford : my suggestion here would be that as a starting point, Ian Jackson and Ch= ris Shepherd work together and work out a minimum environment that works fo= r both OSSTest and XenRT. This would allow us to de-couple the "what are we doing in the long term= " from all the logistical questions. If necessary (and we have enough budge= t), we can extend the setup as needed * Establish concrete requirements for a COLO provider (ideally with a short= -list of possible vendors) * Establish requirements for sysadmin and support * Establish a plan that lists the main tasks that are needed to get us to a= system that is up and running in the short term (I can help with the plan) * Work with all stake-holders on a long-term plan (I can help with the plan= ) B) Ownership of non-technical areas of the WG. I can do this, but would pre= fer if someone else within the group would act as a deputy and work with me= . I travel quite frequently and thus work items may get stuck. The tasks at hand that this WG member will need to drive are : * Work with the community representative and the developer community to ens= ure that what we propose is valuable to the Xen project developers and woul= d in fact be used by them * Handle any contractual issues (e.g. finding and getting a COLO, buying li= censes, buying machines) - in other words act as the interface to the Linux= Foundation * Resolving any issues related to sysadmin and support (e.g. hiring a contr= actor or part-time resource as needed and resolve any practical issues arou= nd it) * Work on proposals for process related questions, such as ** Who can access, how is access managed ** How do we add new HW in future, how do we add extra software configurati= ons ** Etc. =3D=3D=3D Short Term Goal =3D=3D=3D I think we should transition the existing OSSTest system to new hardware im= mediately and then consider how to evolve our test system in the future. We= want to end up with a solution that the community can use to test xen.git,= right away, at our current level of service or better. At the moment only= OSSTest can do this : there are too many open questions related to XenRT a= nd progress on a XenRT proof of concept has been very slow (this has pretty= much stalled since August 2013). However we should choose the infrastructu= re such that both OSSTest and XenRT can run on it. This approach has several advantages * It decouples creating the infrastructure from the choice of test system * It allows us to iron out any logistical and practical issues that owning = an independently hosted test infrastructure involves * It delivers immediate value to the community and shows progress with litt= le disruption. If we stay in limbo, the Xen Project developer community wil= l not write any new tests. Developers will simply wait for a decision. * It leaves all options open, while creating a motivation to work on the L= ong Term Goals and ensure we end up with a solution that works. If we fail = to agree and do not deliver on the Long Term Goals, the Xen Project will st= ill have an independent test infrastructure. However the infrastructure may= be less aligned with the commercial interests of the Advisory Board. In ot= her words, we create a motivation to resolve the long term goals more quick= ly than otherwise would exist. =3D=3D=3D Long Term Goal =3D=3D=3D To ensure that we do the right thing in the long run, I believe we need to = take stock of the technical options we have: right now we have a choice of = OSSTest and XenRT. There may be others as well, which can of course be prop= osed. To figure out what is best in the long run, we need to get a concrete under= standing of * The level of testing that can be achieved with the solutions we have avai= lable today (maybe in form of a very coarse list such as "boot a variety of= different guests", ... for per solution) * The Hardware requirements that are needed to do this * Software requirements (e.g. licenses) * Highlight any issues, work that needs to be performed, etc. BEFORE the so= lution can be deployed * Highlight any concerns that may prevent the Xen Project developer communi= ty from adopting each solution and mitigations on how we may address these (I can help with this and will work with the community representative) * And there may be other areas that I am missing To drive this forward, we need a champion for each proposal who will work w= ith the WG (ideally is a member of it) and who commits to providing the nec= essary information and if necessary secures resources in their respective o= rganization to ensure that gaps and community concerns can be addressed. It= also requires input from the Community rep on the WG. Best Regards Lars ------------------------- Source: Chris Shepherd, Citrix Based on operational expertise with XenRT Item Item Cost Notes One off costs for hardware Core infrastructure (filers, etc.) 24,000.00 A pool of servers and shared storage to run the various VMs you need to run= the network, monitor infrastructure, pxe boot, dhcp, scheduling etc. Osste= st would presumably need something similar. XenRT 'site' infrastructure 12,000.00 XenRT can be built up into multiple 'sites', each consisting of a number of= servers (we have sites of 16, 32 and 64 servers). Each site needs its own = small pool of servers with shared storage and switches to run the site. For= a small XenRT deployment (one site) we could probably share the core infra= structure for the site infrastructure, thereby removing (or at least postpo= ning) this cost. 16 mid range test servers at $4K each 64,000.00 Would only include off-the shelf hardware (no prototypes) Could be lower spec machines (OSSTest machines are at $1K) A reasonable approach would be to use a mixture of machines up to $4K, whic= h should get us to more than 16 servers Total: 100,000.00 Colo cost / month Cost per rack / month 2,500.00 Inclusive of space, power and cooling 20Mbps internet with high SLA / month 1,000.00 Could reduce this cost very significantly in return for lower SLA Sysadmin for HW only 500.00 Based on experience with XenRT, we will probably need 1 day per month for r= outine maintenance, plus time to setup any new equipment. Don't have a mechanism to do this (avg hourly rate $30/h) Conservatively costing this at $500 Total/month: 4,000.00 Remote admin and support of environment 20% of a tools dev /devops type person This is based on Citrix effort to support Citrix' XenRT lab which consists = of well over 200 servers at 2 separate physical sites (US west coast and UK= ). Includes: a) General user support e.g. answering usage queries including helping trou= ble-shoot failed tests (that may be due to infrastructure problems rather t= han product test) b) Clearing out files that are taking up disk space c) Monitoring server & network health and fixing broken machines e.g. serve= r lost BIOS setting, network connectivity etc I don't know the typical cost for such a profile and of course there is the= issue that we cannot employ 20% of a person. An average test engineer sala= ry in the US costs $88000 per annum. It seems that the overhead for OSSTest is higher (Ian Jackson says he spend= s 50% of his time looking after fewer machines) Estimated Total/month: 3,750.00 Conservatively assuming 50% of an engineer for $90K (unloaded) as a ball-pa= rk figure for feasibility purpuses only. This item would need to be revised= after some more discussion within the Advisory Board. --_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi all,

 

please find attached the agenda for the next Test Fr= amework working group meeting.  The meeting is on Feb 13th = at 14:00 UTC. Please check whether you have the invite.

 

The current list of WG members can be found here : <= a href=3D"http://wiki.xenproject.org/wiki/AB_WG/Test_Framework"> http://wiki.xenproject.org/wiki/AB_WG/Test_Framework  If your comp= any is on the Advisory Board, and you would like to influence the direction= of the Test Framework WG, you can still nominate a representative for the = Test Framework WG. Please drop me a line, if you want to join.

 

The good news is that the Advisory Board has approve= d the budget that we asked for (which is outlined below). The total allocat= ed budget for the Test Framework is USD 190,612.50 (including Linux Foundat= ion charges) – this is a conservative estimate. We may be able to build a test system with less that the allocat= ed budget or build a larger system.

 

Note that Konrad Wilk from Oracle cannot attend the = next few WG meetings, and has not found a rep from Oracle to cover for him.= I agreed with Konrad, that we will try and use the mailing lists for decis= ions, reviewing documents, etc. as much as possible and use the meetings to raise concerns and issues. I also= added Ian Jackson to this email thread and the mailing list, as he is main= taining and running OSSTest. OSSTest is the existing test framework for the= Xen Project which is hosted by Citrix and his input is thus important for this group. Ian also nominated = himself as community rep for the WG : we do have to follow the process that= is outlined in the e-mail titled “Looking for a volunteer to represe= nt the Xen Project developer community at Test Working”. If there are no other nominees and no objections b= y the maintainer community, Ian would be confirmed as community representat= ive before the WG meeting.

 

=3D=3D Agenda =3D=3D

 

I would like to review the Strawman of a plan at the= meeting (see below). Feedback by e-mail prior to the meeting is desired. I= f I get feedback before the Test Framework WG meeting I will try to incorpo= rate.

 

At a minimum, I would like to

* Get agreement on some of the basic principles in t= he Strawman

** The split between short term and long term goals<= o:p>

** Agreement by stake-holders, i.e. Ian Jackson and = Chris Shepherd, to provide the information that we need in the short and lo= ng term

** Identify a technical leader : in the best case a = WG member volunteers and will be confirmed by the group.
*** If nobody steps up, we would set a deadline.

*** If the deadline passes and still nobody steps up= , I will need to ask the Advisory Board for advice on how to proceed and wh= ether a shared Test Infrastructure is still viable

** Identify a deputy who will work alongside me on n= on-technical questions

* As a stretch goal, we may be able to approve secti= ons of the plan, or the entire plan

 

=3D=3D Strawman of a Plan to move forward =3D=3D

 

=3D=3D=3D Ownership =3D=3D=3D

 

A) We need someone from this group with technical ex= pertise and insight into testing Xen who can lead and drive technical tasks= in the WG. I cannot fulfil this role and without it, I do not believe we w= ill get anywhere. I am looking for one or several of the vendors on this group to step up. Given that both OS= STest and XenRT are currently the only viable solutions for the test framew= ork on the table, the reality is that Citrix needs to step up. Unless we th= row another solution into the mix.

 

The tasks at hand that the technical leader will nee= d to drive are :

* Describe/specify a viable Hardware set-up that we = can afford :
   my suggestion here would be that as a starting point, Ian Jack= son and Chris Shepherd work together and work out a minimum environment tha= t works for both OSSTest and XenRT.
   This would allow us to de-couple the “what are we d= oing in the long term” from all the logistical questions. If necessar= y (and we have enough budget), we can extend the setup as needed=

* Establish concrete requirements for a COLO provide= r (ideally with a short-list of possible vendors)

* Establish requirements for sysadmin and support

* Establish a plan th= at lists the main tasks that are needed to get us to a system that is up an= d running in the short term (I can help with the plan)
* Work with all stake-holders on a long-term plan (I can help with the plan= )

B) Ownership of non-technical areas of the WG. I can= do this, but would prefer if someone else within the group would act as a = deputy and work with me. I travel quite frequently and thus work items may = get stuck.

 

The tasks at hand that this WG member will need to d= rive are :

* Work with the community representative and the dev= eloper community to ensure that what we propose is valuable to the Xen proj= ect developers and would in fact be used by them

* Handle any contractual issues (e.g. finding and ge= tting a COLO, buying licenses, buying machines) – in other words act = as the interface to the Linux Foundation

* Resolving any issues related to sysadmin and suppo= rt (e.g. hiring a contractor or part-time resource as needed and resolve an= y practical issues around it)

* Work on proposals for process related questions, s= uch as

** Who can access, how is access managed<= /p>

** How do we add new HW in future, how do we add ext= ra software configurations

** Etc.

=3D=3D=3D Short Term Goal =3D=3D=3D

 

I think we should transition the existing OSSTest= system to new hardware immediately and then consider how to evolve our tes= t system in the future. We want to end up with a solution that the communit= y can use to test xen.git, right away, at our current level of service or better.  At the moment only OSSTes= t can do this : there are too many open questions related to XenRT and prog= ress on a XenRT proof of concept has been very slow (this has pretty much s= talled since August 2013). However we should choose the infrastructure such that both OSSTest and XenRT can run = on it.

 

This approach has several advantages

* It decouples creating the infrastructure from t= he choice of test system

* It allows us to iron out any logistical and pra= ctical issues that owning an independently hosted test infrastructure invol= ves

* It delivers immediate value to the community an= d shows progress with little disruption. If we stay in limbo, the Xen Proje= ct developer community will not write any new tests. Developers will simply= wait for a decision.

* It leaves all options open,  while creatin= g a motivation to work on the Long Term Goals and ensure we end up with a s= olution that works. If we fail to agree and do not deliver on the Long Term= Goals, the Xen Project will still have an independent test infrastructure. However the infrastructure may be less= aligned with the commercial interests of the Advisory Board. In other word= s, we create a motivation to resolve the long term goals more quickly than = otherwise would exist.

 

=3D=3D=3D Long Term Goal =3D=3D=3D

 

To ensure that we do = the right thing in the long run, I believe we need to take stock of the tec= hnical options we have: right now we have a choice of OSSTest and XenRT. Th= ere may be others as well, which can of course be proposed.

To figure out what is= best in the long run, we need to get a concrete understanding of

* The level of testing that can be achieved with the= solutions we have available today (maybe in form of a very coarse list suc= h as “boot a variety of different guests”, … for per solu= tion)

* The Hardware requirements that are needed to do th= is

* Software requirements (e.g. licenses)

* Highlight any issues, work that needs to be perfor= med, etc. BEFORE the solution can be deployed

* Highlight any concerns that may prevent the Xen Pr= oject developer community from adopting each solution and mitigations on ho= w we may address these
  (I can help with this and will work with the community represen= tative)
* And there may be other areas that I am missing


To drive this forward, we need a champion for each proposal who will work w= ith the WG (ideally is a member of it) and who commits to providing the nec= essary information and if necessary secures resources in their respective o= rganization to ensure that gaps and community concerns can be addressed. It also requires input from the C= ommunity rep on the WG.

Best Regards

Lars

 

-------------------------

 

Source: Chris = Shepherd, Citrix

Based on opera= tional expertise with XenRT

Item

Item

Cost

Notes

One off cos= ts for hardware

Core infrastru= cture (filers, etc.)

24,000.00

A pool of serv= ers and shared storage to run the various VMs you need to run the network, = monitor infrastructure, pxe boot, dhcp, scheduling etc. Osstest would presumably need something similar.

XenRT 'site' i= nfrastructure

12,000.00

XenRT can be b= uilt up into multiple ‘sites’, each consisting of a number of s= ervers (we have sites of 16, 32 and 64 servers). Each site needs its own small pool of servers with shared storage and switches to run the site= . For a small XenRT deployment (one site) we could probably share the core = infrastructure for the site infrastructure, thereby removing (or at least p= ostponing) this cost.

16 mid range t= est servers at $4K each

64,000.00

Would only inc= lude off-the shelf hardware (no prototypes)
Could be lower spec machines (OSSTest machines are at $1K)
A reasonable approach would be to use a mixture of machines up to $4K, whic= h should get us to more than 16 servers

Total:

100,000.0= 0=

Colo cost /= month

Cost per rack = / month

2,500.00

Inclusive of s= pace, power and cooling

20Mbps interne= t with high SLA / month

1,000.00

Could reduce t= his cost very significantly in return for lower SLA

Sysadmin for H= W only

500.00

= Based on experience with XenRT, we will probably need 1 day per month for r= outine maintenance, plus time to setup any new equipment.
Don’t have a mechanism to do this (avg hourly rate $30/h)
Conservatively costing this at $500

Total/month:

4,000.00= <= /span>

Remote admi= n and support of environment

20% of a tools= dev /devops type person

This is based = on Citrix effort to support Citrix' XenRT lab which consists of well over 2= 00 servers at 2 separate physical sites (US west coast and UK). Includes:
a) General user support e.g. answering usage queries including helping trou= ble-shoot failed tests (that may be due to infrastructure problems rather t= han product test)
b) Clearing out files that are taking up disk space
c) Monitoring server & network health and fixing broken machines e.g. s= erver lost BIOS setting, network connectivity etc

= I don't know the typical cost for such a profile and of course there is the= issue that we cannot employ 20% of a person. An average test engineer salary in the US costs $88000 per annum.

It seems that = the overhead for OSSTest is higher (Ian Jackson says he spends 50% of his t= ime looking after fewer machines)

Estimated Total/month:

3,750.00= <= /span>

= Conservatively assuming 50% of an engineer for $90K (unloaded) as a ball-pa= rk figure for feasibility purpuses only. This item would need to be revised after some more discussion within the Advisory Board.

 

 

--_000_B5EC053C78C5D746BD8AC3386F2303C9127DA8AMSPEX01CL01citri_-- --===============1338677578587307231== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --===============1338677578587307231==-- From wg-test-framework-bounces@lists.xenproject.org Wed Feb 12 19:30:07 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 12 Feb 2014 19:30:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WDfVX-0006q0-FV for archives@lists.xenproject.org; Wed, 12 Feb 2014 19:30:07 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WDfVV-0006pi-Md; Wed, 12 Feb 2014 19:30:06 +0000 Received: from [193.109.254.147:24966] by server-16.bemta-14.messagelabs.com id 12/A3-21945-CBBCBF25; Wed, 12 Feb 2014 19:30:04 +0000 X-Env-Sender: dslutz@verizon.com X-Msg-Ref: server-6.tower-27.messagelabs.com!1392233401!3912034!1 X-Originating-IP: [140.108.26.143] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQwLjEwOC4yNi4xNDMgPT4gMjYwNTMz\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 16884 invoked from network); 12 Feb 2014 19:30:02 -0000 Received: from fldsmtpe04.verizon.com (HELO fldsmtpe04.verizon.com) (140.108.26.143) by server-6.tower-27.messagelabs.com with DHE-RSA-AES256-SHA encrypted SMTP; 12 Feb 2014 19:30:02 -0000 X-IronPort-Anti-Spam-Filtered: false Received: from unknown (HELO fldsmtpi03.verizon.com) ([166.68.71.145]) by fldsmtpe04.verizon.com with ESMTP; 12 Feb 2014 19:30:00 +0000 From: Don Slutz X-VzAPP: 1 X-IronPort-AV: E=Sophos;i="4.95,833,1384300800"; d="scan'208,217";a="652034965" Received: from unknown (HELO don-760.CloudSwitch.com) ([162.47.2.164]) by fldsmtpi03.verizon.com with ESMTP; 12 Feb 2014 19:29:58 +0000 Message-ID: <52FBCBB6.5060608@terremark.com> Date: Wed, 12 Feb 2014 14:29:58 -0500 User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Lars Kurth , "wg-test-framework@lists.xenproject.org" References: In-Reply-To: Cc: "advisory-board@lists.xenproject.org" Subject: Re: [Wg-test-framework] Proposed Agenda for next Test Framework WG meeting X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============6500110543827842754==" Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org This is a multi-part message in MIME format. --===============6500110543827842754== Content-Type: multipart/alternative; boundary="------------060909070107060603010408" This is a multi-part message in MIME format. --------------060909070107060603010408 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 02/07/14 13:27, Lars Kurth wrote: > > Hi all, > > please find attached the agenda for the next Test Framework working gro= up meeting. The meeting is on Feb 13^th at 14:00 UTC. Please check wheth= er you have the invite. > > The current list of WG members can be found here : http://wiki.xenproje= ct.org/wiki/AB_WG/Test_Framework If your company is on the Advisory Board, and you would l= ike to influence the direction of the Test Framework WG, you can still no= minate a representative for the Test Framework WG. Please drop me a line,= if you want to join. > > The good news is that the Advisory Board has approved the budget that w= e asked for (which is outlined below). The total allocated budget for the= Test Framework is USD 190,612.50 (including Linux Foundation charges) --= this is a conservative estimate. We may be able to build a test system w= ith less that the allocated budget or build a larger system. > > Note that Konrad Wilk from Oracle cannot attend the next few WG meeting= s, and has not found a rep from Oracle to cover for him. I agreed with Ko= nrad, that we will try and use the mailing lists for decisions, reviewing= documents, etc. as much as possible and use the meetings to raise concer= ns and issues. I also added Ian Jackson to this email thread and the mail= ing list, as he is maintaining and running OSSTest. OSSTest is the existi= ng test framework for the Xen Project which is hosted by Citrix and his i= nput is thus important for this group. Ian also nominated himself as comm= unity rep for the WG : we do have to follow the process that is outlined = in the e-mail titled "Looking for a volunteer to represent the Xen Projec= t developer community at Test Working". If there are no other nominees an= d no objections by the maintainer community, Ian would be confirmed as co= mmunity representative before the WG meeting. > > =3D=3D Agenda =3D=3D > > I would like to review the Strawman of a plan at the meeting (see below= ). Feedback by e-mail prior to the meeting is desired. If I get feedback = before the Test Framework WG meeting I will try to incorporate. > > At a minimum, I would like to > > * Get agreement on some of the basic principles in the Strawman > > ** The split between short term and long term goals > > ** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to = provide the information that we need in the short and long term > > ** Identify a technical leader : in the best case a WG member volunteer= s and will be confirmed by the group. > I can be a technical leader for this. I can only commit to 8-12 hours a = week for the long term. Most of the time I can do much more, but may nee= d to limit it depending on Verizon needs. -Don Slutz > *** If nobody steps up, we would set a deadline. > > *** If the deadline passes and still nobody steps up, I will need to as= k the Advisory Board for advice on how to proceed and whether a shared Te= st Infrastructure is still viable > > ** Identify a deputy who will work alongside me on non-technical questi= ons > > * As a stretch goal, we may be able to approve sections of the plan, or= the entire plan > > =3D=3D Strawman of a Plan to move forward =3D=3D > > =3D=3D=3D Ownership =3D=3D=3D > > A) We need someone from this group with technical expertise and insight= into testing Xen who can lead and drive technical tasks in the WG. I can= not fulfil this role and without it, I do not believe we will get anywher= e. I am looking for one or several of the vendors on this group to step u= p. Given that both OSSTest and XenRT are currently the only viable soluti= ons for the test framework on the table, the reality is that Citrix needs= to step up. Unless we throw another solution into the mix. > > The tasks at hand that the technical leader will need to drive are : > > * Describe/specify a viable Hardware set-up that we can afford : > my suggestion here would be that as a starting point, Ian Jackson an= d Chris Shepherd work together and work out a minimum environment that wo= rks for both OSSTest and XenRT. > This would allow us to de-couple the "what are we doing in the long = term" from all the logistical questions. If necessary (and we have enough= budget), we can extend the setup as needed > > * Establish concrete requirements for a COLO provider (ideally with a s= hort-list of possible vendors) > > * Establish requirements for sysadmin and support > > * Establish a plan that lists the main tasks that are needed to get us = to a system that is up and running in the short term (I can help with the= plan) > * Work with all stake-holders on a long-term plan (I can help with the = plan) > > B) Ownership of non-technical areas of the WG. I can do this, but would= prefer if someone else within the group would act as a deputy and work w= ith me. I travel quite frequently and thus work items may get stuck. > > The tasks at hand that this WG member will need to drive are : > > * Work with the community representative and the developer community to= ensure that what we propose is valuable to the Xen project developers an= d would in fact be used by them > > * Handle any contractual issues (e.g. finding and getting a COLO, buyin= g licenses, buying machines) -- in other words act as the interface to th= e Linux Foundation > > * Resolving any issues related to sysadmin and support (e.g. hiring a c= ontractor or part-time resource as needed and resolve any practical issue= s around it) > > * Work on proposals for process related questions, such as > > ** Who can access, how is access managed > > ** How do we add new HW in future, how do we add extra software configu= rations > > ** Etc. > > =3D=3D=3D Short Term Goal =3D=3D=3D > > I think we should transition the existing OSSTest system to new hardwar= e immediately and then consider how to evolve our test system in the futu= re. We want to end up with a solution that the community can use to test = xen.git, right away, at our current level of service or better. At the m= oment only OSSTest can do this : there are too many open questions relate= d to XenRT and progress on a XenRT proof of concept has been very slow (t= his has pretty much stalled since August 2013). However we should choose = the infrastructure such that both OSSTest and XenRT can run on it. > > This approach has several advantages > > * It decouples creating the infrastructure from the choice of test syst= em > > * It allows us to iron out any logistical and practical issues that own= ing an independently hosted test infrastructure involves > > * It delivers immediate value to the community and shows progress with = little disruption. If we stay in limbo, the Xen Project developer communi= ty will not write any new tests. Developers will simply wait for a decisi= on. > > * It leaves all options open, while creating a motivation to work on t= he Long Term Goals and ensure we end up with a solution that works. If we= fail to agree and do not deliver on the Long Term Goals, the Xen Project= will still have an independent test infrastructure. However the infrastr= ucture may be less aligned with the commercial interests of the Advisory = Board. In other words, we create a motivation to resolve the long term go= als more quickly than otherwise would exist. > > =3D=3D=3D Long Term Goal =3D=3D=3D > > To ensure that we do the right thing in the long run, I believe we need= to take stock of the technical options we have: right now we have a choi= ce of OSSTest and XenRT. There may be others as well, which can of course= be proposed. > > To figure out what is best in the long run, we need to get a concrete u= nderstanding of > > * The level of testing that can be achieved with the solutions we have = available today (maybe in form of a very coarse list such as "boot a vari= ety of different guests", ... for per solution) > > * The Hardware requirements that are needed to do this > > * Software requirements (e.g. licenses) > > * Highlight any issues, work that needs to be performed, etc. BEFORE th= e solution can be deployed > > * Highlight any concerns that may prevent the Xen Project developer com= munity from adopting each solution and mitigations on how we may address = these > (I can help with this and will work with the community representative= ) > * And there may be other areas that I am missing > > > To drive this forward, we need a champion for each proposal who will wo= rk with the WG (ideally is a member of it) and who commits to providing t= he necessary information and if necessary secures resources in their resp= ective organization to ensure that gaps and community concerns can be add= ressed. It also requires input from the Community rep on the WG. > > Best Regards > > Lars > > ------------------------- > > Source: Chris Shepherd, Citrix > > =09 > =09 > =09 > =09 > > Based on operational expertise with XenRT > > =09 > =09 > =09 > =09 > > =09 > =09 > =09 > =09 > > *Item* > > =09 > > *Item* > > =09 > > *Cost* > > =09 > =09 > > *Notes* > > > =09 > =09 > =09 > =09 > > *One off costs for hardware* > > =09 > > Core infrastructure (filers, etc.) > > =09 > > 24,000.00 > > =09 > =09 > > A pool of servers and shared storage to run the various VMs you need to= run the network, monitor infrastructure, pxe boot, dhcp, scheduling etc.= Osstest would presumably need something similar. > > > =09 > > XenRT 'site' infrastructure > > =09 > > 12,000.00 > > =09 > =09 > > XenRT can be built up into multiple 'sites', each consisting of a numbe= r of servers (we have sites of 16, 32 and 64 servers). Each site needs it= s own small pool of servers with shared storage and switches to run the s= ite. For a small XenRT deployment (one site) we could probably share the = core infrastructure for the site infrastructure, thereby removing (or at = least postponing) this cost. > > > =09 > > 16 mid range test servers at $4K each > > =09 > > 64,000.00 > > =09 > =09 > > Would only include off-the shelf hardware (no prototypes) > Could be lower spec machines (OSSTest machines are at $1K) > A reasonable approach would be to use a mixture of machines up to $4K, = which should get us to more than 16 servers > > > =09 > > *Total:* > > =09 > > 100,000.00 > > =09 > =09 > > =09 > =09 > =09 > =09 > > *Colo cost / month* > > =09 > > Cost per rack / month > > =09 > > 2,500.00 > > =09 > =09 > > Inclusive of space, power and cooling > > > =09 > > 20Mbps internet with high SLA / month > > =09 > > 1,000.00 > > =09 > =09 > > Could reduce this cost very significantly in return for lower SLA > > > =09 > > Sysadmin for HW only > > =09 > > 500.00 > > =09 > =09 > > Based on experience with XenRT, we will probably need 1 day per month f= or routine maintenance, plus time to setup any new equipment. > Don't have a mechanism to do this (avg hourly rate $30/h) > Conservatively costing this at $500 > > > =09 > > *Total/month:* > > =09 > > 4,000.00 > > =09 > =09 > > =09 > =09 > =09 > =09 > > *Remote admin and support of environment* > > =09 > > 20% of a tools dev /devops type person > > =09 > =09 > =09 > > This is based on Citrix effort to support Citrix' XenRT lab which consi= sts of well over 200 servers at 2 separate physical sites (US west coast = and UK). Includes: > a) General user support e.g. answering usage queries including helping = trouble-shoot failed tests (that may be due to infrastructure problems ra= ther than product test) > b) Clearing out files that are taking up disk space > c) Monitoring server & network health and fixing broken machines e.g. s= erver lost BIOS setting, network connectivity etc > > > =09 > =09 > =09 > =09 > > =09 > =09 > =09 > =09 > > I don't know the typical cost for such a profile and of course there is= the issue that we cannot employ 20% of a person. An average test enginee= r salary in the US costs $88000 per annum. > > > =09 > =09 > =09 > =09 > > It seems that the overhead for OSSTest is higher (Ian Jackson says he s= pends 50% of his time looking after fewer machines) > > > =09 > =09 > =09 > =09 > > =09 > > *Estimated Total/month:* > > =09 > > 3,750.00 > > =09 > =09 > > Conservatively assuming 50% of an engineer for $90K (unloaded) as a bal= l-park figure for feasibility purpuses only. This item would need to be r= evised after some more discussion within the Advisory Board. > > > > _______________________________________________ > Wg-test-framework mailing list > Wg-test-framework@lists.xenproject.org > http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --------------060909070107060603010408 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
On 02/07/14 13:27, Lars Kurth wrote:

Hi all,

 

please find attached the agenda for the next Test Framework working group meeting.  The meeting is on Feb 13th at 14:00 UTC. Please check whether you have the invite.

 

The current list of WG members can be found here : http://wiki.xenproject.org/wiki/AB_WG/Test_Framework  If your company is on the Advisory Board, and you would like to influence the direction of the Test Framework WG, you can still nominate a representative for the Test Framework WG. Please drop me a line, if you want to join.

 

The good news is that the Advisory Board has approved the budget that we asked for (which is outlined below). The total allocated budget for the Test Framework is USD 190,612.50 (including Linux Foundation charges) – this is a conservative estimate. We may be able to build a test system with less that the allocated budget or build a larger system.

 

Note that Konrad Wilk from Oracle cannot attend the next few WG meetings, and has not found a rep from Oracle to cover for him. I agreed with Konrad, that we will try and use the mailing lists for decisions, reviewing documents, etc. as much as possible and use the meetings to raise concerns and issues. I also added Ian Jackson to this email thread and the mailing list, as he is maintaining and running OSSTest. OSSTest is the existing test framework for the Xen Project which is hosted by Citrix and his input is thus important for this group. Ian also nominated himself as community rep for the WG : we do have to follow the process that is outlined in the e-mail titled “Looking for a volunteer to represent the Xen Project developer community at Test Working”. If there are no other nominees and no objections by the maintainer community, Ian would be confirmed as community representative before the WG meeting.

 

== Agenda ==

 

I would like to review the Strawman of a plan at the meeting (see below). Feedback by e-mail prior to the meeting is desired. If I get feedback before the Test Framework WG meeting I will try to incorporate.

 

At a minimum, I would like to

* Get agreement on some of the basic principles in the Strawman

** The split between short term and long term goals

** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to provide the information that we need in the short and long term

** Identify a technical leader : in the best case a WG member volunteers and will be confirmed by the group.


I can be a technical leader for this.  I can only commit to 8-12 hours a week for the long term.  Most of the time I can do much more, but may need to limit it depending on Verizon needs.

   -Don Slutz

*** If nobody steps up, we would set a deadline.

*** If the deadline passes and still nobody steps up, I will need to ask the Advisory Board for advice on how to proceed and whether a shared Test Infrastructure is still viable

** Identify a deputy who will work alongside me on non-technical questions

* As a stretch goal, we may be able to approve sections of the plan, or the entire plan

 

== Strawman of a Plan to move forward ==

 

=== Ownership ===

 

A) We need someone from this group with technical expertise and insight into testing Xen who can lead and drive technical tasks in the WG. I cannot fulfil this role and without it, I do not believe we will get anywhere. I am looking for one or several of the vendors on this group to step up. Given that both OSSTest and XenRT are currently the only viable solutions for the test framework on the table, the reality is that Citrix needs to step up. Unless we throw another solution into the mix.

 

The tasks at hand that the technical leader will need to drive are :

* Describe/specify a viable Hardware set-up that we can afford :
   my suggestion here would be that as a starting point, Ian Jackson and Chris Shepherd work together and work out a minimum environment that works for both OSSTest and XenRT.
   This would allow us to de-couple the “what are we doing in the long term” from all the logistical questions. If necessary (and we have enough budget), we can extend the setup as needed

* Establish concrete requirements for a COLO provider (ideally with a short-list of possible vendors)

* Establish requirements for sysadmin and support

* Establish a plan that lists the main tasks that are needed to get us to a system that is up and running in the short term (I can help with the plan)
* Work with all stake-holders on a long-term plan (I can help with the plan)

B) Ownership of non-technical areas of the WG. I can do this, but would prefer if someone else within the group would act as a deputy and work with me. I travel quite frequently and thus work items may get stuck.

 

The tasks at hand that this WG member will need to drive are :

* Work with the community representative and the developer community to ensure that what we propose is valuable to the Xen project developers and would in fact be used by them

* Handle any contractual issues (e.g. finding and getting a COLO, buying licenses, buying machines) – in other words act as the interface to the Linux Foundation

* Resolving any issues related to sysadmin and support (e.g. hiring a contractor or part-time resource as needed and resolve any practical issues around it)

* Work on proposals for process related questions, such as

** Who can access, how is access managed

** How do we add new HW in future, how do we add extra software configurations

** Etc.

=== Short Term Goal ===

 

I think we should transition the existing OSSTest system to new hardware immediately and then consider how to evolve our test system in the future. We want to end up with a solution that the community can use to test xen.git, right away, at our current level of service or better.  At the moment only OSSTest can do this : there are too many open questions related to XenRT and progress on a XenRT proof of concept has been very slow (this has pretty much stalled since August 2013). However we should choose the infrastructure such that both OSSTest and XenRT can run on it.

 

This approach has several advantages

* It decouples creating the infrastructure from the choice of test system

* It allows us to iron out any logistical and practical issues that owning an independently hosted test infrastructure involves

* It delivers immediate value to the community and shows progress with little disruption. If we stay in limbo, the Xen Project developer community will not write any new tests. Developers will simply wait for a decision.

* It leaves all options open,  while creating a motivation to work on the Long Term Goals and ensure we end up with a solution that works. If we fail to agree and do not deliver on the Long Term Goals, the Xen Project will still have an independent test infrastructure. However the infrastructure may be less aligned with the commercial interests of the Advisory Board. In other words, we create a motivation to resolve the long term goals more quickly than otherwise would exist.

 

=== Long Term Goal ===

 

To ensure that we do the right thing in the long run, I believe we need to take stock of the technical options we have: right now we have a choice of OSSTest and XenRT. There may be others as well, which can of course be proposed.

To figure out what is best in the long run, we need to get a concrete understanding of

* The level of testing that can be achieved with the solutions we have available today (maybe in form of a very coarse list such as “boot a variety of different guests”, … for per solution)

* The Hardware requirements that are needed to do this

* Software requirements (e.g. licenses)

* Highlight any issues, work that needs to be performed, etc. BEFORE the solution can be deployed

* Highlight any concerns that may prevent the Xen Project developer community from adopting each solution and mitigations on how we may address these
  (I can help with this and will work with the community representative)
* And there may be other areas that I am missing


To drive this forward, we need a champion for each proposal who will work with the WG (ideally is a member of it) and who commits to providing the necessary information and if necessary secures resources in their respective organization to ensure that gaps and community concerns can be addressed. It also requires input from the Community rep on the WG.

Best Regards

Lars

 

-------------------------

 

Source: Chris Shepherd, Citrix



Based on operational expertise with XenRT








Item

Item

Cost

Notes





One off costs for hardware

Core infrastructure (filers, etc.)

24,000.00

A pool of servers and shared storage to run the various VMs you need to run the network, monitor infrastructure, pxe boot, dhcp, scheduling etc. Osstest would presumably need something similar.

XenRT 'site' infrastructure

12,000.00

XenRT can be built up into multiple ‘sites’, each consisting of a number of servers (we have sites of 16, 32 and 64 servers). Each site needs its own small pool of servers with shared storage and switches to run the site. For a small XenRT deployment (one site) we could probably share the core infrastructure for the site infrastructure, thereby removing (or at least postponing) this cost.

16 mid range test servers at $4K each

64,000.00

Would only include off-the shelf hardware (no prototypes)
Could be lower spec machines (OSSTest machines are at $1K)
A reasonable approach would be to use a mixture of machines up to $4K, which should get us to more than 16 servers

Total:

100,000.00






Colo cost / month

Cost per rack / month

2,500.00

Inclusive of space, power and cooling

20Mbps internet with high SLA / month

1,000.00

Could reduce this cost very significantly in return for lower SLA

Sysadmin for HW only

500.00

Based on experience with XenRT, we will probably need 1 day per month for routine maintenance, plus time to setup any new equipment.
Don’t have a mechanism to do this (avg hourly rate $30/h)
Conservatively costing this at $500

Total/month:

4,000.00






Remote admin and support of environment

20% of a tools dev /devops type person

This is based on Citrix effort to support Citrix' XenRT lab which consists of well over 200 servers at 2 separate physical sites (US west coast and UK). Includes:
a) General user support e.g. answering usage queries including helping trouble-shoot failed tests (that may be due to infrastructure problems rather than product test)
b) Clearing out files that are taking up disk space
c) Monitoring server & network health and fixing broken machines e.g. server lost BIOS setting, network connectivity etc









I don't know the typical cost for such a profile and of course there is the issue that we cannot employ 20% of a person. An average test engineer salary in the US costs $88000 per annum.




It seems that the overhead for OSSTest is higher (Ian Jackson says he spends 50% of his time looking after fewer machines)






Estimated Total/month:

3,750.00

Conservatively assuming 50% of an engineer for $90K (unloaded) as a ball-park figure for feasibility purpuses only. This item would need to be revised after some more discussion within the Advisory Board.

 

 



_______________________________________________
Wg-test-framework mailing list
Wg-test-framework@lists.xenproject.org
http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework

--------------060909070107060603010408-- --===============6500110543827842754== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --===============6500110543827842754==-- From wg-test-framework-bounces@lists.xenproject.org Wed Feb 12 19:30:07 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 12 Feb 2014 19:30:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WDfVX-0006q0-FV for archives@lists.xenproject.org; Wed, 12 Feb 2014 19:30:07 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WDfVV-0006pi-Md; Wed, 12 Feb 2014 19:30:06 +0000 Received: from [193.109.254.147:24966] by server-16.bemta-14.messagelabs.com id 12/A3-21945-CBBCBF25; Wed, 12 Feb 2014 19:30:04 +0000 X-Env-Sender: dslutz@verizon.com X-Msg-Ref: server-6.tower-27.messagelabs.com!1392233401!3912034!1 X-Originating-IP: [140.108.26.143] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQwLjEwOC4yNi4xNDMgPT4gMjYwNTMz\n X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 16884 invoked from network); 12 Feb 2014 19:30:02 -0000 Received: from fldsmtpe04.verizon.com (HELO fldsmtpe04.verizon.com) (140.108.26.143) by server-6.tower-27.messagelabs.com with DHE-RSA-AES256-SHA encrypted SMTP; 12 Feb 2014 19:30:02 -0000 X-IronPort-Anti-Spam-Filtered: false Received: from unknown (HELO fldsmtpi03.verizon.com) ([166.68.71.145]) by fldsmtpe04.verizon.com with ESMTP; 12 Feb 2014 19:30:00 +0000 From: Don Slutz X-VzAPP: 1 X-IronPort-AV: E=Sophos;i="4.95,833,1384300800"; d="scan'208,217";a="652034965" Received: from unknown (HELO don-760.CloudSwitch.com) ([162.47.2.164]) by fldsmtpi03.verizon.com with ESMTP; 12 Feb 2014 19:29:58 +0000 Message-ID: <52FBCBB6.5060608@terremark.com> Date: Wed, 12 Feb 2014 14:29:58 -0500 User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Lars Kurth , "wg-test-framework@lists.xenproject.org" References: In-Reply-To: Cc: "advisory-board@lists.xenproject.org" Subject: Re: [Wg-test-framework] Proposed Agenda for next Test Framework WG meeting X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============6500110543827842754==" Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org This is a multi-part message in MIME format. --===============6500110543827842754== Content-Type: multipart/alternative; boundary="------------060909070107060603010408" This is a multi-part message in MIME format. --------------060909070107060603010408 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 02/07/14 13:27, Lars Kurth wrote: > > Hi all, > > please find attached the agenda for the next Test Framework working gro= up meeting. The meeting is on Feb 13^th at 14:00 UTC. Please check wheth= er you have the invite. > > The current list of WG members can be found here : http://wiki.xenproje= ct.org/wiki/AB_WG/Test_Framework If your company is on the Advisory Board, and you would l= ike to influence the direction of the Test Framework WG, you can still no= minate a representative for the Test Framework WG. Please drop me a line,= if you want to join. > > The good news is that the Advisory Board has approved the budget that w= e asked for (which is outlined below). The total allocated budget for the= Test Framework is USD 190,612.50 (including Linux Foundation charges) --= this is a conservative estimate. We may be able to build a test system w= ith less that the allocated budget or build a larger system. > > Note that Konrad Wilk from Oracle cannot attend the next few WG meeting= s, and has not found a rep from Oracle to cover for him. I agreed with Ko= nrad, that we will try and use the mailing lists for decisions, reviewing= documents, etc. as much as possible and use the meetings to raise concer= ns and issues. I also added Ian Jackson to this email thread and the mail= ing list, as he is maintaining and running OSSTest. OSSTest is the existi= ng test framework for the Xen Project which is hosted by Citrix and his i= nput is thus important for this group. Ian also nominated himself as comm= unity rep for the WG : we do have to follow the process that is outlined = in the e-mail titled "Looking for a volunteer to represent the Xen Projec= t developer community at Test Working". If there are no other nominees an= d no objections by the maintainer community, Ian would be confirmed as co= mmunity representative before the WG meeting. > > =3D=3D Agenda =3D=3D > > I would like to review the Strawman of a plan at the meeting (see below= ). Feedback by e-mail prior to the meeting is desired. If I get feedback = before the Test Framework WG meeting I will try to incorporate. > > At a minimum, I would like to > > * Get agreement on some of the basic principles in the Strawman > > ** The split between short term and long term goals > > ** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to = provide the information that we need in the short and long term > > ** Identify a technical leader : in the best case a WG member volunteer= s and will be confirmed by the group. > I can be a technical leader for this. I can only commit to 8-12 hours a = week for the long term. Most of the time I can do much more, but may nee= d to limit it depending on Verizon needs. -Don Slutz > *** If nobody steps up, we would set a deadline. > > *** If the deadline passes and still nobody steps up, I will need to as= k the Advisory Board for advice on how to proceed and whether a shared Te= st Infrastructure is still viable > > ** Identify a deputy who will work alongside me on non-technical questi= ons > > * As a stretch goal, we may be able to approve sections of the plan, or= the entire plan > > =3D=3D Strawman of a Plan to move forward =3D=3D > > =3D=3D=3D Ownership =3D=3D=3D > > A) We need someone from this group with technical expertise and insight= into testing Xen who can lead and drive technical tasks in the WG. I can= not fulfil this role and without it, I do not believe we will get anywher= e. I am looking for one or several of the vendors on this group to step u= p. Given that both OSSTest and XenRT are currently the only viable soluti= ons for the test framework on the table, the reality is that Citrix needs= to step up. Unless we throw another solution into the mix. > > The tasks at hand that the technical leader will need to drive are : > > * Describe/specify a viable Hardware set-up that we can afford : > my suggestion here would be that as a starting point, Ian Jackson an= d Chris Shepherd work together and work out a minimum environment that wo= rks for both OSSTest and XenRT. > This would allow us to de-couple the "what are we doing in the long = term" from all the logistical questions. If necessary (and we have enough= budget), we can extend the setup as needed > > * Establish concrete requirements for a COLO provider (ideally with a s= hort-list of possible vendors) > > * Establish requirements for sysadmin and support > > * Establish a plan that lists the main tasks that are needed to get us = to a system that is up and running in the short term (I can help with the= plan) > * Work with all stake-holders on a long-term plan (I can help with the = plan) > > B) Ownership of non-technical areas of the WG. I can do this, but would= prefer if someone else within the group would act as a deputy and work w= ith me. I travel quite frequently and thus work items may get stuck. > > The tasks at hand that this WG member will need to drive are : > > * Work with the community representative and the developer community to= ensure that what we propose is valuable to the Xen project developers an= d would in fact be used by them > > * Handle any contractual issues (e.g. finding and getting a COLO, buyin= g licenses, buying machines) -- in other words act as the interface to th= e Linux Foundation > > * Resolving any issues related to sysadmin and support (e.g. hiring a c= ontractor or part-time resource as needed and resolve any practical issue= s around it) > > * Work on proposals for process related questions, such as > > ** Who can access, how is access managed > > ** How do we add new HW in future, how do we add extra software configu= rations > > ** Etc. > > =3D=3D=3D Short Term Goal =3D=3D=3D > > I think we should transition the existing OSSTest system to new hardwar= e immediately and then consider how to evolve our test system in the futu= re. We want to end up with a solution that the community can use to test = xen.git, right away, at our current level of service or better. At the m= oment only OSSTest can do this : there are too many open questions relate= d to XenRT and progress on a XenRT proof of concept has been very slow (t= his has pretty much stalled since August 2013). However we should choose = the infrastructure such that both OSSTest and XenRT can run on it. > > This approach has several advantages > > * It decouples creating the infrastructure from the choice of test syst= em > > * It allows us to iron out any logistical and practical issues that own= ing an independently hosted test infrastructure involves > > * It delivers immediate value to the community and shows progress with = little disruption. If we stay in limbo, the Xen Project developer communi= ty will not write any new tests. Developers will simply wait for a decisi= on. > > * It leaves all options open, while creating a motivation to work on t= he Long Term Goals and ensure we end up with a solution that works. If we= fail to agree and do not deliver on the Long Term Goals, the Xen Project= will still have an independent test infrastructure. However the infrastr= ucture may be less aligned with the commercial interests of the Advisory = Board. In other words, we create a motivation to resolve the long term go= als more quickly than otherwise would exist. > > =3D=3D=3D Long Term Goal =3D=3D=3D > > To ensure that we do the right thing in the long run, I believe we need= to take stock of the technical options we have: right now we have a choi= ce of OSSTest and XenRT. There may be others as well, which can of course= be proposed. > > To figure out what is best in the long run, we need to get a concrete u= nderstanding of > > * The level of testing that can be achieved with the solutions we have = available today (maybe in form of a very coarse list such as "boot a vari= ety of different guests", ... for per solution) > > * The Hardware requirements that are needed to do this > > * Software requirements (e.g. licenses) > > * Highlight any issues, work that needs to be performed, etc. BEFORE th= e solution can be deployed > > * Highlight any concerns that may prevent the Xen Project developer com= munity from adopting each solution and mitigations on how we may address = these > (I can help with this and will work with the community representative= ) > * And there may be other areas that I am missing > > > To drive this forward, we need a champion for each proposal who will wo= rk with the WG (ideally is a member of it) and who commits to providing t= he necessary information and if necessary secures resources in their resp= ective organization to ensure that gaps and community concerns can be add= ressed. It also requires input from the Community rep on the WG. > > Best Regards > > Lars > > ------------------------- > > Source: Chris Shepherd, Citrix > > =09 > =09 > =09 > =09 > > Based on operational expertise with XenRT > > =09 > =09 > =09 > =09 > > =09 > =09 > =09 > =09 > > *Item* > > =09 > > *Item* > > =09 > > *Cost* > > =09 > =09 > > *Notes* > > > =09 > =09 > =09 > =09 > > *One off costs for hardware* > > =09 > > Core infrastructure (filers, etc.) > > =09 > > 24,000.00 > > =09 > =09 > > A pool of servers and shared storage to run the various VMs you need to= run the network, monitor infrastructure, pxe boot, dhcp, scheduling etc.= Osstest would presumably need something similar. > > > =09 > > XenRT 'site' infrastructure > > =09 > > 12,000.00 > > =09 > =09 > > XenRT can be built up into multiple 'sites', each consisting of a numbe= r of servers (we have sites of 16, 32 and 64 servers). Each site needs it= s own small pool of servers with shared storage and switches to run the s= ite. For a small XenRT deployment (one site) we could probably share the = core infrastructure for the site infrastructure, thereby removing (or at = least postponing) this cost. > > > =09 > > 16 mid range test servers at $4K each > > =09 > > 64,000.00 > > =09 > =09 > > Would only include off-the shelf hardware (no prototypes) > Could be lower spec machines (OSSTest machines are at $1K) > A reasonable approach would be to use a mixture of machines up to $4K, = which should get us to more than 16 servers > > > =09 > > *Total:* > > =09 > > 100,000.00 > > =09 > =09 > > =09 > =09 > =09 > =09 > > *Colo cost / month* > > =09 > > Cost per rack / month > > =09 > > 2,500.00 > > =09 > =09 > > Inclusive of space, power and cooling > > > =09 > > 20Mbps internet with high SLA / month > > =09 > > 1,000.00 > > =09 > =09 > > Could reduce this cost very significantly in return for lower SLA > > > =09 > > Sysadmin for HW only > > =09 > > 500.00 > > =09 > =09 > > Based on experience with XenRT, we will probably need 1 day per month f= or routine maintenance, plus time to setup any new equipment. > Don't have a mechanism to do this (avg hourly rate $30/h) > Conservatively costing this at $500 > > > =09 > > *Total/month:* > > =09 > > 4,000.00 > > =09 > =09 > > =09 > =09 > =09 > =09 > > *Remote admin and support of environment* > > =09 > > 20% of a tools dev /devops type person > > =09 > =09 > =09 > > This is based on Citrix effort to support Citrix' XenRT lab which consi= sts of well over 200 servers at 2 separate physical sites (US west coast = and UK). Includes: > a) General user support e.g. answering usage queries including helping = trouble-shoot failed tests (that may be due to infrastructure problems ra= ther than product test) > b) Clearing out files that are taking up disk space > c) Monitoring server & network health and fixing broken machines e.g. s= erver lost BIOS setting, network connectivity etc > > > =09 > =09 > =09 > =09 > > =09 > =09 > =09 > =09 > > I don't know the typical cost for such a profile and of course there is= the issue that we cannot employ 20% of a person. An average test enginee= r salary in the US costs $88000 per annum. > > > =09 > =09 > =09 > =09 > > It seems that the overhead for OSSTest is higher (Ian Jackson says he s= pends 50% of his time looking after fewer machines) > > > =09 > =09 > =09 > =09 > > =09 > > *Estimated Total/month:* > > =09 > > 3,750.00 > > =09 > =09 > > Conservatively assuming 50% of an engineer for $90K (unloaded) as a bal= l-park figure for feasibility purpuses only. This item would need to be r= evised after some more discussion within the Advisory Board. > > > > _______________________________________________ > Wg-test-framework mailing list > Wg-test-framework@lists.xenproject.org > http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --------------060909070107060603010408 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
On 02/07/14 13:27, Lars Kurth wrote:

Hi all,

 

please find attached the agenda for the next Test Framework working group meeting.  The meeting is on Feb 13th at 14:00 UTC. Please check whether you have the invite.

 

The current list of WG members can be found here : http://wiki.xenproject.org/wiki/AB_WG/Test_Framework  If your company is on the Advisory Board, and you would like to influence the direction of the Test Framework WG, you can still nominate a representative for the Test Framework WG. Please drop me a line, if you want to join.

 

The good news is that the Advisory Board has approved the budget that we asked for (which is outlined below). The total allocated budget for the Test Framework is USD 190,612.50 (including Linux Foundation charges) – this is a conservative estimate. We may be able to build a test system with less that the allocated budget or build a larger system.

 

Note that Konrad Wilk from Oracle cannot attend the next few WG meetings, and has not found a rep from Oracle to cover for him. I agreed with Konrad, that we will try and use the mailing lists for decisions, reviewing documents, etc. as much as possible and use the meetings to raise concerns and issues. I also added Ian Jackson to this email thread and the mailing list, as he is maintaining and running OSSTest. OSSTest is the existing test framework for the Xen Project which is hosted by Citrix and his input is thus important for this group. Ian also nominated himself as community rep for the WG : we do have to follow the process that is outlined in the e-mail titled “Looking for a volunteer to represent the Xen Project developer community at Test Working”. If there are no other nominees and no objections by the maintainer community, Ian would be confirmed as community representative before the WG meeting.

 

== Agenda ==

 

I would like to review the Strawman of a plan at the meeting (see below). Feedback by e-mail prior to the meeting is desired. If I get feedback before the Test Framework WG meeting I will try to incorporate.

 

At a minimum, I would like to

* Get agreement on some of the basic principles in the Strawman

** The split between short term and long term goals

** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to provide the information that we need in the short and long term

** Identify a technical leader : in the best case a WG member volunteers and will be confirmed by the group.


I can be a technical leader for this.  I can only commit to 8-12 hours a week for the long term.  Most of the time I can do much more, but may need to limit it depending on Verizon needs.

   -Don Slutz

*** If nobody steps up, we would set a deadline.

*** If the deadline passes and still nobody steps up, I will need to ask the Advisory Board for advice on how to proceed and whether a shared Test Infrastructure is still viable

** Identify a deputy who will work alongside me on non-technical questions

* As a stretch goal, we may be able to approve sections of the plan, or the entire plan

 

== Strawman of a Plan to move forward ==

 

=== Ownership ===

 

A) We need someone from this group with technical expertise and insight into testing Xen who can lead and drive technical tasks in the WG. I cannot fulfil this role and without it, I do not believe we will get anywhere. I am looking for one or several of the vendors on this group to step up. Given that both OSSTest and XenRT are currently the only viable solutions for the test framework on the table, the reality is that Citrix needs to step up. Unless we throw another solution into the mix.

 

The tasks at hand that the technical leader will need to drive are :

* Describe/specify a viable Hardware set-up that we can afford :
   my suggestion here would be that as a starting point, Ian Jackson and Chris Shepherd work together and work out a minimum environment that works for both OSSTest and XenRT.
   This would allow us to de-couple the “what are we doing in the long term” from all the logistical questions. If necessary (and we have enough budget), we can extend the setup as needed

* Establish concrete requirements for a COLO provider (ideally with a short-list of possible vendors)

* Establish requirements for sysadmin and support

* Establish a plan that lists the main tasks that are needed to get us to a system that is up and running in the short term (I can help with the plan)
* Work with all stake-holders on a long-term plan (I can help with the plan)

B) Ownership of non-technical areas of the WG. I can do this, but would prefer if someone else within the group would act as a deputy and work with me. I travel quite frequently and thus work items may get stuck.

 

The tasks at hand that this WG member will need to drive are :

* Work with the community representative and the developer community to ensure that what we propose is valuable to the Xen project developers and would in fact be used by them

* Handle any contractual issues (e.g. finding and getting a COLO, buying licenses, buying machines) – in other words act as the interface to the Linux Foundation

* Resolving any issues related to sysadmin and support (e.g. hiring a contractor or part-time resource as needed and resolve any practical issues around it)

* Work on proposals for process related questions, such as

** Who can access, how is access managed

** How do we add new HW in future, how do we add extra software configurations

** Etc.

=== Short Term Goal ===

 

I think we should transition the existing OSSTest system to new hardware immediately and then consider how to evolve our test system in the future. We want to end up with a solution that the community can use to test xen.git, right away, at our current level of service or better.  At the moment only OSSTest can do this : there are too many open questions related to XenRT and progress on a XenRT proof of concept has been very slow (this has pretty much stalled since August 2013). However we should choose the infrastructure such that both OSSTest and XenRT can run on it.

 

This approach has several advantages

* It decouples creating the infrastructure from the choice of test system

* It allows us to iron out any logistical and practical issues that owning an independently hosted test infrastructure involves

* It delivers immediate value to the community and shows progress with little disruption. If we stay in limbo, the Xen Project developer community will not write any new tests. Developers will simply wait for a decision.

* It leaves all options open,  while creating a motivation to work on the Long Term Goals and ensure we end up with a solution that works. If we fail to agree and do not deliver on the Long Term Goals, the Xen Project will still have an independent test infrastructure. However the infrastructure may be less aligned with the commercial interests of the Advisory Board. In other words, we create a motivation to resolve the long term goals more quickly than otherwise would exist.

 

=== Long Term Goal ===

 

To ensure that we do the right thing in the long run, I believe we need to take stock of the technical options we have: right now we have a choice of OSSTest and XenRT. There may be others as well, which can of course be proposed.

To figure out what is best in the long run, we need to get a concrete understanding of

* The level of testing that can be achieved with the solutions we have available today (maybe in form of a very coarse list such as “boot a variety of different guests”, … for per solution)

* The Hardware requirements that are needed to do this

* Software requirements (e.g. licenses)

* Highlight any issues, work that needs to be performed, etc. BEFORE the solution can be deployed

* Highlight any concerns that may prevent the Xen Project developer community from adopting each solution and mitigations on how we may address these
  (I can help with this and will work with the community representative)
* And there may be other areas that I am missing


To drive this forward, we need a champion for each proposal who will work with the WG (ideally is a member of it) and who commits to providing the necessary information and if necessary secures resources in their respective organization to ensure that gaps and community concerns can be addressed. It also requires input from the Community rep on the WG.

Best Regards

Lars

 

-------------------------

 

Source: Chris Shepherd, Citrix



Based on operational expertise with XenRT








Item

Item

Cost

Notes





One off costs for hardware

Core infrastructure (filers, etc.)

24,000.00

A pool of servers and shared storage to run the various VMs you need to run the network, monitor infrastructure, pxe boot, dhcp, scheduling etc. Osstest would presumably need something similar.

XenRT 'site' infrastructure

12,000.00

XenRT can be built up into multiple ‘sites’, each consisting of a number of servers (we have sites of 16, 32 and 64 servers). Each site needs its own small pool of servers with shared storage and switches to run the site. For a small XenRT deployment (one site) we could probably share the core infrastructure for the site infrastructure, thereby removing (or at least postponing) this cost.

16 mid range test servers at $4K each

64,000.00

Would only include off-the shelf hardware (no prototypes)
Could be lower spec machines (OSSTest machines are at $1K)
A reasonable approach would be to use a mixture of machines up to $4K, which should get us to more than 16 servers

Total:

100,000.00






Colo cost / month

Cost per rack / month

2,500.00

Inclusive of space, power and cooling

20Mbps internet with high SLA / month

1,000.00

Could reduce this cost very significantly in return for lower SLA

Sysadmin for HW only

500.00

Based on experience with XenRT, we will probably need 1 day per month for routine maintenance, plus time to setup any new equipment.
Don’t have a mechanism to do this (avg hourly rate $30/h)
Conservatively costing this at $500

Total/month:

4,000.00






Remote admin and support of environment

20% of a tools dev /devops type person

This is based on Citrix effort to support Citrix' XenRT lab which consists of well over 200 servers at 2 separate physical sites (US west coast and UK). Includes:
a) General user support e.g. answering usage queries including helping trouble-shoot failed tests (that may be due to infrastructure problems rather than product test)
b) Clearing out files that are taking up disk space
c) Monitoring server & network health and fixing broken machines e.g. server lost BIOS setting, network connectivity etc









I don't know the typical cost for such a profile and of course there is the issue that we cannot employ 20% of a person. An average test engineer salary in the US costs $88000 per annum.




It seems that the overhead for OSSTest is higher (Ian Jackson says he spends 50% of his time looking after fewer machines)






Estimated Total/month:

3,750.00

Conservatively assuming 50% of an engineer for $90K (unloaded) as a ball-park figure for feasibility purpuses only. This item would need to be revised after some more discussion within the Advisory Board.

 

 



_______________________________________________
Wg-test-framework mailing list
Wg-test-framework@lists.xenproject.org
http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework

--------------060909070107060603010408-- --===============6500110543827842754== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --===============6500110543827842754==-- From wg-test-framework-bounces@lists.xenproject.org Tue Feb 18 11:56:26 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 18 Feb 2014 11:56:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WFjHm-0003BO-10 for archives@lists.xenproject.org; Tue, 18 Feb 2014 11:56:26 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WFjHk-0003B3-IY; Tue, 18 Feb 2014 11:56:24 +0000 Received: from [85.158.139.211:13576] by server-15.bemta-5.messagelabs.com id AB/2B-24395-76A43035; Tue, 18 Feb 2014 11:56:23 +0000 X-Env-Sender: lars.kurth@citrix.com X-Msg-Ref: server-14.tower-206.messagelabs.com!1392724581!4620179!1 X-Originating-IP: [185.25.65.24] X-SpamReason: No, hits=0.6 required=7.0 tests=BODY_RANDOM_LONG, HTML_50_60,HTML_MESSAGE X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 16286 invoked from network); 18 Feb 2014 11:56:22 -0000 Received: from smtp.ctxuk.citrix.com (HELO SMTP.EU.CITRIX.COM) (185.25.65.24) by server-14.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 18 Feb 2014 11:56:22 -0000 X-IronPort-AV: E=Sophos;i="4.97,501,1389744000"; d="scan'208,217";a="9620912" Received: from unknown (HELO AMSPEX01CL02.citrite.net) ([10.69.60.9]) by AMSPIP01.EU.Citrix.com with ESMTP; 18 Feb 2014 11:56:22 +0000 Received: from AMSPEX01CL01.citrite.net ([169.254.6.60]) by AMSPEX01CL02.citrite.net ([10.69.46.33]) with mapi id 14.02.0342.004; Tue, 18 Feb 2014 12:56:21 +0100 From: Lars Kurth To: "wg-test-framework@lists.xenproject.org" Thread-Topic: [Minutes] February Test WG meeting Thread-Index: Ac8soGcMIaoWR+KGTuyvE6viJfwvwg== Date: Tue, 18 Feb 2014 11:56:20 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.2.168] MIME-Version: 1.0 X-DLP: AMS1 Cc: "advisory-board@lists.xenproject.org" Subject: [Wg-test-framework] [Minutes] February Test WG meeting X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============8434386595143357532==" Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org --===============8434386595143357532== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_" --_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable =3D=3D Attendees =3D=3D * Aravind Gopalakrishnan (AMD, voting) * Chris Shepherd (Citrix, voting) * Doug Clark (Citrix, helping Chris with detailed technical tasks) * Ian Jackson (Citrix, community representative, voting) * Don Slutz (Verizon, voting) * Don Dugger (Intel, voting) =3D=3D Agenda =3D=3D See http://lists.xenproject.org/archives/html/wg-test-framework/2014-02/msg= 00002.html * Get agreement on some of the basic principles in the draft plan (strawman= ) ** The split between short term and long term goals ** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to prov= ide the information that we need in the short and long term ** Identify a technical leader : in the best case a WG member volunteers an= d will be confirmed by the group. *** If nobody steps up, we would set a deadline. *** If the deadline passes and still nobody steps up, I will need to ask th= e Advisory Board for advice on how to proceed and whether a shared Test Inf= rastructure is still viable ** Identify a deputy who will work alongside me on non-technical questions * As a stretch goal, we may be able to approve sections of the plan, or the= entire plan =3D=3D=3D Agree basic principles of plan =3D=3D=3D The group has agreed to the plan as outlined in http://lists.xenproject.org= /archives/html/wg-test-framework/2014-02/msg00002.html Aravind asked whether we are still planning to get XenRT running on OUSL. Chris apologized for the slow progress and will look into this. I think the= plan is still to do this. =3D=3D=3D Technical Leadership =3D=3D=3D Don Slutz stepped up as technical leader and coordinator for the group. The= group confirmed Don. Ian Jackson and Doug Clark will work closely with Don= . We will use the mailing list for further discussion. In summary: Doug and Ian will collaborate to define the community use-cases= and the sw test framework we will put in place to meet them - both in the = short term and with options for longer term evolution (possibly to meet dif= ferent use-cases etc). They will share know-how on osstest and xenrt. We do= not believe that the hw infrastructure requirements will differ between th= ese two scenarios. This info will be used to spec out the hw infrastructure= requirements. They will send these to Don, who will lead the effort to def= ine and implement the lab solution including sys admin. {{Action|Ian + Doug Clark}} Send a list of hw requirements covering both os= stest and xenrt use-cases to the list for further review before the next me= eting We also had a discussion about development hw (Intel stated they could make= some available). We all agreed that development hw (rather than production= hw) should not be covered in the short term regression test environment. B= ut it would be a valuable longer term goal to grant access to development h= ardware. {{Action|Don}} Start collating a list of long-term requirements and issues = on the Xen Project wiki. Add the requirement of access to "development boxe= s" to it We then had a discussion about several current issues ... =3D=3D=3D=3D Sysadmin Support =3D=3D=3D=3D * Ian raised the issue that sysadmin support is too thinly spread * The big issue is how we turn the "budget" into a sysadmin resource. We ha= ve several options: ** Some Colo's sell percentage of a time of a person as part of the package= . Don knows a number of Colo's which do this. This would include installing= machines, software on machines, serviced backups, etc. ** Other - less desirable options were - to employ a sysadmin through a Xen= [project AB member company {{Action|Don}} Investigate Colo options for support and report back to the = group =3D=3D=3D=3D Required Initial Capacity =3D=3D=3D=3D Chris and Doug were asking what the load of the current test OSSTEST system= is. * Ian: we have 11 machines and test staging branches and xen-unstable. Curr= ently a test run takes 6-8 hours and there is always a delay of several hou= rs when changes are tested via the pushgate. In other words we have a bottl= eneck today. * After some discussion, we came to the conclusion that demand will likely = grow as we add extra capacity. We concluded that the short term plan should cover 2-3 times the capacity = we have now. The tentative agreement was that this should be doable under o= r within the budget. We can then grow as needed. {{Action|Ian}} Once we have some of the details (aka Actions above) resolve= , Ian will send the agreed hw spec to the developer lists (xen-devel, xen-a= pi and mirageos) for review and input =3D=3D=3D=3D Holiday cover =3D=3D=3D=3D Lars will be on holiday for the next meeting. Chris volunteered to chair th= e next meeting {{Action|Lars}} Send Chris call details {{Action|Lars}} Chris to agree agenda and post meeting minutes for March me= eting --_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

=3D=3D Attendees =3D=3D

* Aravind Gopalakrishnan = (AMD, voting)
* Chris Shepherd (Citrix, voting)
* Doug Clark (Citrix, helping Chris with detailed technical tasks)
* Ian Jackson (Citrix, community representative, voting)
* Don Slutz (Verizon, voting)
* Don Dugger (Intel, voting)

 

=3D=3D Agenda =3D=3D

See http://lists.xenproject.org/archives/html/wg-test-framework/2014-02/msg0000= 2.html

 

* Get agreement on some of the basic principles in t= he draft plan (strawman)

** The split between short term and long term goals<= o:p>

** Agreement by stake-holders, i.e. Ian Jackson and = Chris Shepherd, to provide the information that we need in the short and lo= ng term

** Identify a technical leader : in the best case a = WG member volunteers and will be confirmed by the group.

*** If nobody steps up, we would set a deadline.

*** If the deadline passes and still nobody steps up= , I will need to ask the Advisory Board for advice on how to proceed and wh= ether a shared Test Infrastructure is still viable

** Identify a deputy who will work alongside me on n= on-technical questions

* As a stretch goal, we may be able to approve secti= ons of the plan, or the entire plan

 

=3D=3D=3D Agree basic principles of plan =3D=3D=3D

The group has agreed to the plan as outlined in http://lists.xenproject.org/archives/html/wg-test-framework/2014-02/msg0000= 2.html

 

Aravind asked whether we are still planning to get X= enRT running on OUSL.

Chris apologized for the slow progress and will look= into this. I think the plan is still to do this.

 

=3D=3D=3D Technical Leadership =3D=3D=3D<= /p>

Don Slutz stepped up as technical leader and coordin= ator for the group. The group confirmed Don. Ian Jackson and Doug Clark wil= l work closely with Don. We will use the mailing list for further discussio= n.

 

In summar= y: Doug and Ian will collaborate to define the community use-cases and the = sw test framework we will put in place to meet them – both in the sho= rt term and with options for longer term evolution (possibly to meet different use-cases etc). They will share know-how on os= stest and xenrt. We do not believe that the hw infrastructure requirements = will differ between these two scenarios. This info will be used to spec out= the hw infrastructure requirements. They will send these to Don, who will lead the effort to define and implem= ent the lab solution including sys admin.

 

{{Action|Ian + Doug Clark}} Send a list of hw re= quirements covering both osstest and xenrt use-cases to the list for furthe= r review before the next meeting

 

We also had a discussion about development hw (Intel= stated they could make some available). We all agreed that development hw = (rather than production hw) should not be covered in the short term regress= ion test environment. But it would be a valuable longer term goal to grant access to development hardware.

 

{{Action|Don}} Start collating a list of long-term r= equirements and issues on the Xen Project wiki. Add the requirement of acce= ss to “development boxes“ to it

 

We then had a discussion about several current issue= s …

 

=3D=3D=3D=3D Sysadmin Support =3D=3D=3D=3D

 

* Ian raised the issue that sysadmin support is too = thinly spread

* The big issue is how we turn the “budgetR= 21; into a sysadmin resource. We have several options:

** Some Colo’s sell percentage of a time of a = person as part of the package. Don knows a number of Colo’s which do = this. This would include installing machines, software on machines, service= d backups, etc.

** Other – less desirable options were –= to employ a sysadmin through a Xen [project AB member company

 

{{Action|Don}} Investigate Colo options for support = and report back to the group

 

=3D=3D=3D=3D Required Initial Capacity =3D=3D=3D=3D<= o:p>

 

Chris and Doug were asking what the load of the curr= ent test OSSTEST system is.

* Ian: we have 11 machines and test staging branches= and xen-unstable. Currently a test run takes 6-8 hours and there is always= a delay of several hours when changes are tested via the pushgate. In othe= r words we have a bottleneck today.

* After some discussion, we came to the conclusion t= hat demand will likely grow as we add extra capacity.

 

We concluded that the short term plan should  c= over 2-3 times the capacity we have now. The tentative agreement was that t= his should be doable under or within the budget. We can then grow as needed= .

 

{{Action|Ian}} Once we have some of the details (aka= Actions above) resolve, Ian will send the agreed hw spec to the developer = lists (xen-devel, xen-api and mirageos) for review and input

 

=3D=3D=3D=3D Holiday cover =3D=3D=3D=3D

Lars will be on holiday for the next meeting. Chris = volunteered to chair the next meeting

 

{{Action|Lars}} Send Chris call details

{{Action|Lars}} Chris to agree agenda and post meeti= ng minutes for March meeting

 

 

--_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_-- --===============8434386595143357532== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --===============8434386595143357532==-- From wg-test-framework-bounces@lists.xenproject.org Tue Feb 18 11:56:26 2014 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 18 Feb 2014 11:56:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WFjHm-0003BO-10 for archives@lists.xenproject.org; Tue, 18 Feb 2014 11:56:26 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WFjHk-0003B3-IY; Tue, 18 Feb 2014 11:56:24 +0000 Received: from [85.158.139.211:13576] by server-15.bemta-5.messagelabs.com id AB/2B-24395-76A43035; Tue, 18 Feb 2014 11:56:23 +0000 X-Env-Sender: lars.kurth@citrix.com X-Msg-Ref: server-14.tower-206.messagelabs.com!1392724581!4620179!1 X-Originating-IP: [185.25.65.24] X-SpamReason: No, hits=0.6 required=7.0 tests=BODY_RANDOM_LONG, HTML_50_60,HTML_MESSAGE X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 16286 invoked from network); 18 Feb 2014 11:56:22 -0000 Received: from smtp.ctxuk.citrix.com (HELO SMTP.EU.CITRIX.COM) (185.25.65.24) by server-14.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 18 Feb 2014 11:56:22 -0000 X-IronPort-AV: E=Sophos;i="4.97,501,1389744000"; d="scan'208,217";a="9620912" Received: from unknown (HELO AMSPEX01CL02.citrite.net) ([10.69.60.9]) by AMSPIP01.EU.Citrix.com with ESMTP; 18 Feb 2014 11:56:22 +0000 Received: from AMSPEX01CL01.citrite.net ([169.254.6.60]) by AMSPEX01CL02.citrite.net ([10.69.46.33]) with mapi id 14.02.0342.004; Tue, 18 Feb 2014 12:56:21 +0100 From: Lars Kurth To: "wg-test-framework@lists.xenproject.org" Thread-Topic: [Minutes] February Test WG meeting Thread-Index: Ac8soGcMIaoWR+KGTuyvE6viJfwvwg== Date: Tue, 18 Feb 2014 11:56:20 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.2.168] MIME-Version: 1.0 X-DLP: AMS1 Cc: "advisory-board@lists.xenproject.org" Subject: [Wg-test-framework] [Minutes] February Test WG meeting X-BeenThere: wg-test-framework@lists.xenproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen Project Test Framework Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============8434386595143357532==" Sender: wg-test-framework-bounces@lists.xenproject.org Errors-To: wg-test-framework-bounces@lists.xenproject.org --===============8434386595143357532== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_" --_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable =3D=3D Attendees =3D=3D * Aravind Gopalakrishnan (AMD, voting) * Chris Shepherd (Citrix, voting) * Doug Clark (Citrix, helping Chris with detailed technical tasks) * Ian Jackson (Citrix, community representative, voting) * Don Slutz (Verizon, voting) * Don Dugger (Intel, voting) =3D=3D Agenda =3D=3D See http://lists.xenproject.org/archives/html/wg-test-framework/2014-02/msg= 00002.html * Get agreement on some of the basic principles in the draft plan (strawman= ) ** The split between short term and long term goals ** Agreement by stake-holders, i.e. Ian Jackson and Chris Shepherd, to prov= ide the information that we need in the short and long term ** Identify a technical leader : in the best case a WG member volunteers an= d will be confirmed by the group. *** If nobody steps up, we would set a deadline. *** If the deadline passes and still nobody steps up, I will need to ask th= e Advisory Board for advice on how to proceed and whether a shared Test Inf= rastructure is still viable ** Identify a deputy who will work alongside me on non-technical questions * As a stretch goal, we may be able to approve sections of the plan, or the= entire plan =3D=3D=3D Agree basic principles of plan =3D=3D=3D The group has agreed to the plan as outlined in http://lists.xenproject.org= /archives/html/wg-test-framework/2014-02/msg00002.html Aravind asked whether we are still planning to get XenRT running on OUSL. Chris apologized for the slow progress and will look into this. I think the= plan is still to do this. =3D=3D=3D Technical Leadership =3D=3D=3D Don Slutz stepped up as technical leader and coordinator for the group. The= group confirmed Don. Ian Jackson and Doug Clark will work closely with Don= . We will use the mailing list for further discussion. In summary: Doug and Ian will collaborate to define the community use-cases= and the sw test framework we will put in place to meet them - both in the = short term and with options for longer term evolution (possibly to meet dif= ferent use-cases etc). They will share know-how on osstest and xenrt. We do= not believe that the hw infrastructure requirements will differ between th= ese two scenarios. This info will be used to spec out the hw infrastructure= requirements. They will send these to Don, who will lead the effort to def= ine and implement the lab solution including sys admin. {{Action|Ian + Doug Clark}} Send a list of hw requirements covering both os= stest and xenrt use-cases to the list for further review before the next me= eting We also had a discussion about development hw (Intel stated they could make= some available). We all agreed that development hw (rather than production= hw) should not be covered in the short term regression test environment. B= ut it would be a valuable longer term goal to grant access to development h= ardware. {{Action|Don}} Start collating a list of long-term requirements and issues = on the Xen Project wiki. Add the requirement of access to "development boxe= s" to it We then had a discussion about several current issues ... =3D=3D=3D=3D Sysadmin Support =3D=3D=3D=3D * Ian raised the issue that sysadmin support is too thinly spread * The big issue is how we turn the "budget" into a sysadmin resource. We ha= ve several options: ** Some Colo's sell percentage of a time of a person as part of the package= . Don knows a number of Colo's which do this. This would include installing= machines, software on machines, serviced backups, etc. ** Other - less desirable options were - to employ a sysadmin through a Xen= [project AB member company {{Action|Don}} Investigate Colo options for support and report back to the = group =3D=3D=3D=3D Required Initial Capacity =3D=3D=3D=3D Chris and Doug were asking what the load of the current test OSSTEST system= is. * Ian: we have 11 machines and test staging branches and xen-unstable. Curr= ently a test run takes 6-8 hours and there is always a delay of several hou= rs when changes are tested via the pushgate. In other words we have a bottl= eneck today. * After some discussion, we came to the conclusion that demand will likely = grow as we add extra capacity. We concluded that the short term plan should cover 2-3 times the capacity = we have now. The tentative agreement was that this should be doable under o= r within the budget. We can then grow as needed. {{Action|Ian}} Once we have some of the details (aka Actions above) resolve= , Ian will send the agreed hw spec to the developer lists (xen-devel, xen-a= pi and mirageos) for review and input =3D=3D=3D=3D Holiday cover =3D=3D=3D=3D Lars will be on holiday for the next meeting. Chris volunteered to chair th= e next meeting {{Action|Lars}} Send Chris call details {{Action|Lars}} Chris to agree agenda and post meeting minutes for March me= eting --_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

=3D=3D Attendees =3D=3D

* Aravind Gopalakrishnan = (AMD, voting)
* Chris Shepherd (Citrix, voting)
* Doug Clark (Citrix, helping Chris with detailed technical tasks)
* Ian Jackson (Citrix, community representative, voting)
* Don Slutz (Verizon, voting)
* Don Dugger (Intel, voting)

 

=3D=3D Agenda =3D=3D

See http://lists.xenproject.org/archives/html/wg-test-framework/2014-02/msg0000= 2.html

 

* Get agreement on some of the basic principles in t= he draft plan (strawman)

** The split between short term and long term goals<= o:p>

** Agreement by stake-holders, i.e. Ian Jackson and = Chris Shepherd, to provide the information that we need in the short and lo= ng term

** Identify a technical leader : in the best case a = WG member volunteers and will be confirmed by the group.

*** If nobody steps up, we would set a deadline.

*** If the deadline passes and still nobody steps up= , I will need to ask the Advisory Board for advice on how to proceed and wh= ether a shared Test Infrastructure is still viable

** Identify a deputy who will work alongside me on n= on-technical questions

* As a stretch goal, we may be able to approve secti= ons of the plan, or the entire plan

 

=3D=3D=3D Agree basic principles of plan =3D=3D=3D

The group has agreed to the plan as outlined in http://lists.xenproject.org/archives/html/wg-test-framework/2014-02/msg0000= 2.html

 

Aravind asked whether we are still planning to get X= enRT running on OUSL.

Chris apologized for the slow progress and will look= into this. I think the plan is still to do this.

 

=3D=3D=3D Technical Leadership =3D=3D=3D<= /p>

Don Slutz stepped up as technical leader and coordin= ator for the group. The group confirmed Don. Ian Jackson and Doug Clark wil= l work closely with Don. We will use the mailing list for further discussio= n.

 

In summar= y: Doug and Ian will collaborate to define the community use-cases and the = sw test framework we will put in place to meet them – both in the sho= rt term and with options for longer term evolution (possibly to meet different use-cases etc). They will share know-how on os= stest and xenrt. We do not believe that the hw infrastructure requirements = will differ between these two scenarios. This info will be used to spec out= the hw infrastructure requirements. They will send these to Don, who will lead the effort to define and implem= ent the lab solution including sys admin.

 

{{Action|Ian + Doug Clark}} Send a list of hw re= quirements covering both osstest and xenrt use-cases to the list for furthe= r review before the next meeting

 

We also had a discussion about development hw (Intel= stated they could make some available). We all agreed that development hw = (rather than production hw) should not be covered in the short term regress= ion test environment. But it would be a valuable longer term goal to grant access to development hardware.

 

{{Action|Don}} Start collating a list of long-term r= equirements and issues on the Xen Project wiki. Add the requirement of acce= ss to “development boxes“ to it

 

We then had a discussion about several current issue= s …

 

=3D=3D=3D=3D Sysadmin Support =3D=3D=3D=3D

 

* Ian raised the issue that sysadmin support is too = thinly spread

* The big issue is how we turn the “budgetR= 21; into a sysadmin resource. We have several options:

** Some Colo’s sell percentage of a time of a = person as part of the package. Don knows a number of Colo’s which do = this. This would include installing machines, software on machines, service= d backups, etc.

** Other – less desirable options were –= to employ a sysadmin through a Xen [project AB member company

 

{{Action|Don}} Investigate Colo options for support = and report back to the group

 

=3D=3D=3D=3D Required Initial Capacity =3D=3D=3D=3D<= o:p>

 

Chris and Doug were asking what the load of the curr= ent test OSSTEST system is.

* Ian: we have 11 machines and test staging branches= and xen-unstable. Currently a test run takes 6-8 hours and there is always= a delay of several hours when changes are tested via the pushgate. In othe= r words we have a bottleneck today.

* After some discussion, we came to the conclusion t= hat demand will likely grow as we add extra capacity.

 

We concluded that the short term plan should  c= over 2-3 times the capacity we have now. The tentative agreement was that t= his should be doable under or within the budget. We can then grow as needed= .

 

{{Action|Ian}} Once we have some of the details (aka= Actions above) resolve, Ian will send the agreed hw spec to the developer = lists (xen-devel, xen-api and mirageos) for review and input

 

=3D=3D=3D=3D Holiday cover =3D=3D=3D=3D

Lars will be on holiday for the next meeting. Chris = volunteered to chair the next meeting

 

{{Action|Lars}} Send Chris call details

{{Action|Lars}} Chris to agree agenda and post meeti= ng minutes for March meeting

 

 

--_000_B5EC053C78C5D746BD8AC3386F2303C9142A30AMSPEX01CL01citri_-- --===============8434386595143357532== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Wg-test-framework mailing list Wg-test-framework@lists.xenproject.org http://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-test-framework --===============8434386595143357532==--