Re: build broken + doc

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 15 Sep 2011, at 16:51, Richard Mortier wrote:

> 
> On 15 Sep 2011, at 16:46, Anil Madhavapeddy wrote:
>> so once I'm back from Japan after next week, I'd be very keen to hook it 
>> into a continuous build system so we can keep an eye on the numbers as we 
>> experiment with changes.
> 
> would be nice if the regress build system allowed one to express a range 
> against some performance target.  eg., have a postrun script that extracted 
> the relevant qps figure from dnsperf output, and then ensured that it was 
> within N% of some specified target.  i'm just trying to work my way through 
> your new external target changes etc- is that the kind of thing that should 
> already be supported though?

Balraj also wanted something similar for his Lwt tests --- there was a sleep 
call, and a micro benchmark to ensure that the time that actually passed is 
within the correct parameters (+- a certain number of ms depending on the 
backend).

The only thing that happens right now is that a shell script runs before and 
after (with arguments 'prerun' and 'post run'), and so this could be the place 
where we do the checks. Within the Mirage libraries, I've added an 
Lwt-compliant oUnit port (mirage/lib/oUnit.mli), so that could be a good 
starting point for adding unit assertions.

We need to think about the best way to structure output like performance 
numbers separately from the usual logging output a bit more...

Anil