[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Xen-devel] Call to write tests for osstest
On 09/08/13 13:00, Wei Liu wrote:
On Fri, Aug 09, 2013 at 01:46:42PM +0200, Sander Eikelenboom wrote:
Friday, August 9, 2013, 1:19:54 PM, you wrote:
On gio, 2013-08-08 at 19:32 +0200, Sander Eikelenboom wrote:
* Some performance testing (network, block, cpu/mem/fork/real apps benchmarks,
some other metrics)
- perhaps makes separate graphs of these, so one can see performance
increase or decrease over a larger time frame, and see at around what commits
that occurred.
- only after all basic tests succeeded and a push was done.
We are after this too. I am looking at how to make it possible to do
something like that on top of OSSTest.
Perf benchmarking is a little bit different from regression
smoke-testing, but still I think (hope? :-)) that most of the
infrastructure can be reused.
Yes it does require to keep the rest of the circumstances the same for a longer
period of time.
But it could be quite valuable, slow introduced and minor performance
regressions are hard to discover.
Also, what hardware to use and how to properly schedule these kind of
"tests" is something that needs a bit more of thinking/discussion, I
think.
Yes since you have to keep the "environment" the same, the machine should only
run the perf tests at that time.
This worries me. It is really hard to keep the "environment" the same.
AIUI network / block etc. performance can be affected by other kernel
subsystems. Also the kernel config could also have impact on
performance.
The same is true for functional regressions -- a bug could be introduced
by Xen, qemu, or the kernel. The tester already keeps track of this, as
far as I know, and only moves one thing at a time.
-George
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
