Re: Test framework thoughts

[Richard Mortier <Richard.Mortier@xxxxxxxxxxxxxxxx>]

On 14 Aug 2011, at 15:46, Anil Madhavapeddy wrote:
> Right, but the problem here is that you have to link in *all* the tests into 
> one huge binary, which takes quite a while to compile. It's a similar problem 
> to the tests in our ORM library.
> 
> On the other hand, if each test is a module, then ocamlbuild can link up a 
> binary to run just that one test (with a target like 'ocamlbuild test1.run'), 
> and that would pull in any dependencies automatically.

not sure if this question makes sense but - is it reasonable to have a mirage 
program that only succeeds for a subset of the backends?

> You could still have a 'run all.itarget' that would include all the tests for 
> a full run, but this way I could just quickly define some oUnit tests for a 
> module like JSON, and test it without having to worry about all the others 
> tests running.
> 
>> For the complex settings to run some of the unit-tests, I guess you need to 
>> have a library able to set-up correctly you environment, and use register 
>> some post and pre-hooks if needed.
> 
> Indeed... some of the settings are very specific to the backend being used 
> (like the Xen ones), so it's a bit tricky to do this from one big library.
> 
> The other thing with ocamlbuild is that it can run the tests in parallel 
> without any additional help, as '-j' should just work if the dependencies are 
> setup correctly.  Any errors would show up as a build failure, and the 
> product file would be the performance results.

neat; even better would be an ability to assert that the results file produce 
had to meet a template configuration, so you couldn't validly commit code that 
broke performance.

-- 
Cheers,

R.

This message and any attachment are intended solely for the addressee and may 
contain confidential information. If you have received this message in error, 
please send it back to me, and immediately delete it.   Please do not use, copy 
or disclose the information contained in this message or in any attachment.  
Any views or opinions expressed by the author of this email do not necessarily 
reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an attachment
may still contain software viruses which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.