Hello. > > > >>>>> [...] > >>>>> > >>>>> The idea is to have "interleaved" calls to the candidate > >>>>> implementations, so > >>>>> that (hopefully) they will be penalized (or benefit) in the same > >>>>> way > >>>>> by what > >>>>> the JVM is doing (GC or JIT compilation or ...) while the > >>>>> benchmark > >>>>> is > >>>>> running. > >>>>> > >>>>> Does this make sense? > >>>> Could it be merged by the FastMath performance tests Sebb set up ? > >>> I don't think so. If you meant rewriting the > >>> "FastMathTestPerformance" tests > >>> using the proposed utility, I don't think that it is necessary. > >> This was what I meant. > >> If this feature is not used in any existing tests, perhaps it should go in > >> some other directory. Perhaps a new "utilities" or something like that, at > >> the > >> same level as "main" and "test" ? > >> > >> Anyway, if you feel it's useful to have this available around, don't > >> hesitate. > >> > > Well, the first use I had in mind was to provide a agreed on way to base a > > discussion for requests such as "CM's implementation of foo is not > > efficient", and avoid wondering how the reporter got his results. [This > > problem occurs for the MATH-628 issue.]. Then, when the problem reported > > is confirmed, the new implementation will replace the less efficient one in > > CM, so that there won't be any alternative implementation left to compare. > > > > If you agree with the idea of a "standard" benchmark, it would be very much > > necessary that several people have a look at the code: It might be that my > > crude "methodology" is not right, or that there is a bug. > > > > If the code is accepted, then we'll decide where to put it. Even if, > > according to the above, its primary use will not be for long-lived unit > > tests, it might still be useful in order to compare the efficiency of CM's > > algorithms such as the various optimizers. These comparisons could be added > > as performance reports similar to "FastMathTestPerformance". > > +1 to include it. I would say start by putting it in a top level > package of its own - say, "benchmark" in src/test/java. That way we > can use it in test classes or experimentation that we do using test > classes to set up benchmarks.
Isn't it "safer" to put it in package "o.a.c.m" (under "src/test/java")? I was thinking of "PerfTestUtils" for the class name. > If it evolves into a generically > useful microbenchmark generator, we can talk about moving it > src/main. Thanks for doing this. I didn't think that this utility would ever move to "main", as it's just for internal testing. Regards, Gilles --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
