On 10/18/12 11:13 PM, Sébastien Brisard wrote: > Hi, > I have recently started to update the users guide on special > functions, providing accuracy measurements of our implementations. > To get these figures, I carried out extensive comparisons with > reference values computed with Maxima, and 128 decimal digits. > I also wrote a Java command-line app to automatize the process. > Briefly speaking, the reference values computed with Maxima are saved > as a binary file > - for a unary function f(x), the data is stored as follows > x[0], f(x[0]), x[1], f(x[1]), ... > - for a binary function f(x, y), the data is stored as follows > x[0], y[0], f(x[0], y[0]), x[1], y[1], f(x[1], y[1]), ... > - and similar storage pattern for a n-ary function. > > The signature of the function to be tested can be arbitrary, provided > all its arguments are of primitive type: the app will manage to read > the reference values. > The app then computes for each t-uple (x[i], y[i], ...) the > Commons-Math value of f(x[i], y[i], ...) and the error in ulps. This > error is transmitted to a SummaryStatistics, which is printed on the > standard output when the end of the input file is reached. > The app also writes a binary output file, where the data is stored as follows > x[0], y[0], reference value of f(x[0], y[0]), CM value of f(x[0], > y[0]), error in ulps, ... > > this binary file can then be plotted if necessary in order to locate > the areas where the accuracy is at its worst. > > The app takes a properties file as input, here is an example > > method=org.apache.commons.math3.special.Gamma.logGamma > signature=double > inputFileMask=logGamma-%02d.dat > outputFileMask=logGamma-out-%02d.dat > from=1 > to=5 > by=1 > > The "method" key is the fully qualified name to the function to be > validated. Requirements on this function are > - static > - returns double > - takes only primitive arguments > > The "signature" key is necessary to distinguish between functions with > same name. In case there are multiple arguments, the value of this > key should read e.g. "double, double" > > "inputFileMask" and "outputFileMask" are the file names of the input > and output binary files. In order to be able to handle multiple files > in a row, indexed file names can be used, the format for the indexed > file names must then follow the syntax of String.format(). > "from" is the value of the first index (inclusive), "to" is the value > of the last index (exclusive), "by" is the increment. > > This app is very simple, but it could prove useful to anyone > implementing a new special function in CM. Therefore, I was wondering > what would be the best way to include it in our library. Also, I would > like people to be able to check all the figures I state on the > website. Therefore, I would like to provide all the reference data > I've used so far (I have more in store, not yet used to update the > users guide). As previously discussed, I gave up binary files for unit > tests, which are just "safety guards" to check whether or not the > implementation is totally wrong. However, for this extensive analysis > of the accuracy, I thought it was better to stick with binary data > files. > > What do you think? Do you think that this app should be provided to > all? Same question for reference data files [1]? > > Thanks for your comments, > Sébastien > > [1] For reference data files, maybe providing the Maxima scripts (and > the properties files) would suffice.
I would go ahead and commit all of this stuff in test/maxima, similarly to what we have now for R. It would be great to also have similar tests using R, as R is freely available OSS and having two different comparison impls would help avoid "we both made the same mistake" issues or false negatives. Make sure to include a text file in the top level test/maxima directory that describes how everything works, so others can add patches. Thanks for doing this. Phil > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
