Re: [Xen-devel] Call to write tests for osstest

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Fri, Aug 09, 2013 at 01:46:42PM +0200, Sander Eikelenboom wrote:
> 
> Friday, August 9, 2013, 1:19:54 PM, you wrote:
> 
> > On gio, 2013-08-08 at 19:32 +0200, Sander Eikelenboom wrote:
> >> * Some performance testing (network, block, cpu/mem/fork/real apps 
> >> benchmarks, some other metrics)
> >>    - perhaps makes separate graphs of these, so one can see performance 
> >> increase or decrease over a larger time frame, and see at around what 
> >> commits that occurred.
> >>    - only after all basic tests succeeded and a push was done.
> >> 
> > We are after this too. I am looking at how to make it possible to do
> > something like that on top of OSSTest.
> 
> > Perf benchmarking is a little bit different from regression
> > smoke-testing, but still I think (hope? :-)) that most of the
> > infrastructure can be reused.
> 
> Yes it does require to keep the rest of the circumstances the same for a 
> longer period of time.
> But it could be quite valuable, slow introduced and minor performance 
> regressions are hard to discover.
> 
> > Also, what hardware to use and how to properly schedule these kind of
> > "tests" is something that needs a bit more of thinking/discussion, I
> > think.
> 
> Yes since you have to keep the "environment" the same, the machine should 
> only run the perf tests at that time.
> 

This worries me. It is really hard to keep the "environment" the same.
AIUI network / block etc. performance can be affected by other kernel
subsystems. Also the kernel config could also have impact on
performance.


Wei.

> > Anyway, although I'm not committing to have all it 100% ready for 4.4,
> > George, feel free to add this to your tracking list and put m name on
> > it.
> 
> > Regards,
> > Dario
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel