On 7/26/11 3:52 AM, Gilles Sadowski wrote: > Hello. > >>>>> [...] >>>>> >>>>> The idea is to have "interleaved" calls to the candidate >>>>> implementations, so >>>>> that (hopefully) they will be penalized (or benefit) in the same >>>>> way >>>>> by what >>>>> the JVM is doing (GC or JIT compilation or ...) while the >>>>> benchmark >>>>> is >>>>> running. >>>>> >>>>> Does this make sense? >>>> Could it be merged by the FastMath performance tests Sebb set up ? >>> I don't think so. If you meant rewriting the >>> "FastMathTestPerformance" tests >>> using the proposed utility, I don't think that it is necessary. >> This was what I meant. >> If this feature is not used in any existing tests, perhaps it should go in >> some other directory. Perhaps a new "utilities" or something like that, at >> the >> same level as "main" and "test" ? >> >> Anyway, if you feel it's useful to have this available around, don't >> hesitate. >> > Well, the first use I had in mind was to provide a agreed on way to base a > discussion for requests such as "CM's implementation of foo is not > efficient", and avoid wondering how the reporter got his results. [This > problem occurs for the MATH-628 issue.]. Then, when the problem reported > is confirmed, the new implementation will replace the less efficient one in > CM, so that there won't be any alternative implementation left to compare. > > If you agree with the idea of a "standard" benchmark, it would be very much > necessary that several people have a look at the code: It might be that my > crude "methodology" is not right, or that there is a bug. > > If the code is accepted, then we'll decide where to put it. Even if, > according to the above, its primary use will not be for long-lived unit > tests, it might still be useful in order to compare the efficiency of CM's > algorithms such as the various optimizers. These comparisons could be added > as performance reports similar to "FastMathTestPerformance".
+1 to include it. I would say start by putting it in a top level package of its own - say, "benchmark" in src/test/java. That way we can use it in test classes or experimentation that we do using test classes to set up benchmarks. If it evolves into a generically useful microbenchmark generator, we can talk about moving it src/main. Thanks for doing this. Phil > > > Thanks, > Gilles > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
