Nikolaus Hansen, Luc and me discussed this issue in Toulouse. We have two options to handle this kind of failure in tests of stochastic optimization algorithms: 1) fixed random seed - but this reduces the value of the test 2) Using the RetryRunner - preferred solution
@Retry(3) should be sufficient for all tests. Shall I prepare a patch? > @RunWith(RetryRunner.class) before the corresponding test class and by adding the annotation @Retry(n) with any some > small integer, say 3 to 5 -----Ursprüngliche Nachricht----- Von: Luc Maisonobe [mailto:luc.maison...@free.fr] Gesendet: Donnerstag, 28. April 2011 10:40 An: Commons Developers List Betreff: [math] [GUMP@vmgump]: Project commons-math (in module apache-commons) failed Le 28/04/2011 00:43, Gump a écrit : > To whom it may engage... > > This is an automated request, but not an unsolicited one. For more > information please visit http://gump.apache.org/nagged.html, > and/or contact the folk at gene...@gump.apache.org. > > Project commons-math has an issue affecting its community integration. > This issue affects 1 projects, > and has been outstanding for 7 runs. > The current state of this project is 'Failed', with reason 'Build Failed'. > For reference only, the following projects are affected by this: > - commons-math : The Jakarta Mathematics Library > > > Full details are available at: > > http://vmgump.apache.org/gump/public/apache-commons/commons-math/index > .html Hi all, Once again, the failure is due to some tests in CMA-ES. The output of the failed test is here: <http://vmgump.apache.org/gump/public/apache-commons/commons-math/gump_file/ TEST-org.apache.commons.math.optimization.direct.CMAESOptimizerTest.txt.html >. Dietmar, could you check this and tell if the test failure is expected or not ? If the test should fail once every few runs (typically due to some random drawings within the test), we have a way to handle such failures in the automatic tests. We run the tests that may fail a few times to make sure we can separate random failures from systematic failures. The way we do this is by adding the annotation @RunWith(RetryRunner.class) before the corresponding test class and by adding the annotation @Retry(n) with any some small integer, say 3 to 5, before the test method (in addition to the classical JUnit @Test annotation). The number n is the number of retries we allow before declaring the test systematically failed. Could you have a look at this ? thanks Luc > > That said, some information snippets are provided here. > > The following annotations (debug/informational/warning/error messages) were provided: > -DEBUG- Sole jar output [commons-math-28042011.jar] identifier set to > project name > -DEBUG- Dependency on junit exists, no need to add for property junit.jar. > -INFO- Failed with reason build failed > -INFO- Project Reports in: > /srv/gump/public/workspace/apache-commons/math/target/surefire-reports > -WARNING- No directory > [/srv/gump/public/workspace/apache-commons/math/target/surefire-report > s] > -INFO- Project Reports in: > /srv/gump/public/workspace/apache-commons/math/target/test-reports > -INFO- Failed to extract fallback artifacts from Gump Repository > > > > The following work was performed: > http://vmgump.apache.org/gump/public/apache-commons/commons-math/gump_ > work/build_apache-commons_commons-math.html > Work Name: build_apache-commons_commons-math (Type: Build) Work ended > in a state of : Failed > Elapsed: 5 mins 18 secs > Command Line: /usr/lib/jvm/java-6-openjdk/bin/java > -Djava.awt.headless=true -Dbuild.sysclasspath=only > -Xbootclasspath/p:/srv/gump/public/workspace/xml-xerces2/build/xercesI > mpl.jar:/srv/gump/public/workspace/xml-commons/java/external/build/xml > -apis.jar org.apache.tools.ant.Main > -Dgump.merge=/srv/gump/public/gump/work/merge.xml > -Djunit.jar=/srv/gump/public/workspace/junit/dist/junit-28042011.jar > -Dfinal.name=commons-math-28042011 jar [Working Directory: > /srv/gump/public/workspace/apache-commons/math] > CLASSPATH: > /usr/lib/jvm/java-6-openjdk/lib/tools.jar:/srv/gump/public/workspace/a > pache-commons/math/target/classes:/srv/gump/public/workspace/apache-co > mmons/math/target/test-classes:/srv/gump/public/workspace/ant/dist/lib > /ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv > /gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/works > pace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/li > b/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-res > olver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.ja > r:/srv/gump/packages/junit3.8.1/junit.jar:/srv/gump/public/workspace/x > ml-commons/java/build/resolver.jar:/srv/gump/public/workspace/xml-comm > ons/java/external/build/xml-apis-ext.jar:/srv/gump/public/workspace/ju > nit/dist/junit-28042011.jar:/srv/gump/public/workspace/junit/dist/juni > t-dep-28042011.jar > --------------------------------------------- > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.17 sec > [junit] Running org.apache.commons.math.stat.ranking.NaturalRankingTest > [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.157 sec > [junit] Running org.apache.commons.math.stat.regression.GLSMultipleLinearRegressionTest > [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 1.064 sec > [junit] Running org.apache.commons.math.stat.regression.OLSMultipleLinearRegressionTest > [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 0.295 sec > [junit] Running org.apache.commons.math.stat.regression.SimpleRegressionTest > [junit] Tests run: 16, Failures: 0, Errors: 0, Time elapsed: 0.202 sec > [junit] Running org.apache.commons.math.transform.FastCosineTransformerTest > [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.155 sec > [junit] Running org.apache.commons.math.transform.FastFourierTransformerTest > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.178 sec > [junit] Running org.apache.commons.math.transform.FastHadamardTransformerTest > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.153 sec > [junit] Running org.apache.commons.math.transform.FastSineTransformerTest > [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.173 sec > [junit] Running org.apache.commons.math.util.BigRealFieldTest > [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.03 sec > [junit] Running org.apache.commons.math.util.BigRealTest > [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 0.064 sec > [junit] Running org.apache.commons.math.util.ContinuedFractionTest > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.136 sec > [junit] Running org.apache.commons.math.util.DefaultTransformerTest > [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.041 sec > [junit] Running org.apache.commons.math.util.FastMathStrictComparisonTest > [junit] Tests run: 53, Failures: 0, Errors: 0, Time elapsed: 0.481 sec > [junit] Running org.apache.commons.math.util.FastMathTest > [junit] Tests run: 35, Failures: 0, Errors: 0, Time elapsed: 15.605 sec > [junit] Running org.apache.commons.math.util.IncrementorTest > [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.026 sec > [junit] Running org.apache.commons.math.util.MathUtilsTest > [junit] Tests run: 73, Failures: 0, Errors: 0, Time elapsed: 1.918 sec > [junit] Running org.apache.commons.math.util.MultidimensionalCounterTest > [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.17 sec > [junit] Running org.apache.commons.math.util.OpenIntToDoubleHashMapTest > [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.423 sec > [junit] Running org.apache.commons.math.util.OpenIntToFieldTest > [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.274 sec > [junit] Running org.apache.commons.math.util.PairTest > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.016 sec > [junit] Running org.apache.commons.math.util.ResizableDoubleArrayTest > [junit] Tests run: 14, Failures: 0, Errors: 0, Time elapsed: 0.202 sec > [junit] Running org.apache.commons.math.util.TransformerMapTest > [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 0.039 > sec > > BUILD FAILED > /srv/gump/public/workspace/apache-commons/math/build.xml:214: There were test failures. > > Total time: 5 minutes 18 seconds > --------------------------------------------- > > To subscribe to this information via syndicated feeds: > - RSS: > http://vmgump.apache.org/gump/public/apache-commons/commons-math/rss.x > ml > - Atom: > http://vmgump.apache.org/gump/public/apache-commons/commons-math/atom. > xml > > ============================== Gump Tracking Only === Produced by > Apache Gump(TM) version 2.3. > Gump Run 10000028042011, vmgump.apache.org:vmgump:10000028042011 > Gump E-mail Identifier (unique within run) #3. > > -- > Apache Gump > http://gump.apache.org/ [Instance: vmgump] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
