Understood. Thanks, -Rob > On Jul 8, 2024, at 4:41 PM, Alex Herbert <alex.d.herb...@gmail.com> wrote: > > Hi Rob, > > This behaviour is documented in the release note. > > The tests for randomness use a random seed. So they fail with a given > probability. We have a set number of reruns configured. A few failures get > logged as a flake. If they fail 3 times then the build will fail. With a > p-value of 0.001 this will be 1 in a billion. But there are thousands of > tests. The build fail rate is closer to 1 in 1000 but I haven't done the > analysis on it for a while. > > Hope that helps. > > Alex > > > > > On Mon, 8 Jul 2024, 21:30 Rob Tompkins, <chtom...@gmail.com> wrote: > >> I was running the build on java 21 and got this: >> Results: >> [INFO] >> [WARNING] Flakes: >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextBytesPartialBuffer(UniformRandomProvider)[8] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextBytesPartialBuffer:124 >> org.apache.commons.rng.core.source32.Well44497b ==> expected: <true> but >> was: <false> >> [INFO] Run 2: PASS >> [INFO] >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextFloat(UniformRandomProvider)[25] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextFloat:167->checkNextFloat:418->checkNextInRange:528 >> org.apache.commons.rng.core.source32.XoShiRo128PlusPlus: Too many failures >> for n = 1234, sample size = 1000 (12 out of 500 tests failed, chi2 > >> 21.666=[25.107, 26.161, 22.291, 21.885, 23.420, 32.030, 23.780, 21.808, >> 22.042, 22.531, 28.331, 25.669]) >> [INFO] Run 2: PASS >> [INFO] >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextFloat(UniformRandomProvider)[30] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextFloat:167->checkNextFloat:418->checkNextInRange:528 >> org.apache.commons.rng.core.source64.TwoCmres (Cmres: [0xedce446814d3b3d9L, >> 33, 330658535] + Cmres: [0xc5b3cf786c806df7L, 33, 331932042]): Too many >> failures for n = 1234, sample size = 1000 (14 out of 500 tests failed, chi2 >>> 21.666=[22.259, 23.260, 22.523, 30.913, 23.325, 22.569, 32.086, 24.000, >> 22.940, 23.656, 39.190, 23.015, 23.873, 24.451]) >> [INFO] Run 2: PASS >> [INFO] >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextIntegerInRange(UniformRandomProvider)[25] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextIntegerInRange:142->checkNextIntegerInRange:389->checkNextInRange:528 >> org.apache.commons.rng.core.source32.XoShiRo128PlusPlus: Too many failures >> for n = 32, sample size = 1000 (13 out of 500 tests failed, chi2 > >> 21.666=[23.824, 25.531, 22.309, 23.461, 23.315, 22.693, 21.968, 21.749, >> 30.803, 23.157, 26.877, 22.016, 21.896]) >> [INFO] Run 2: PASS >> [INFO] >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextIntegerInRange(UniformRandomProvider)[2] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextIntegerInRange:139->checkNextIntegerInRange:389->checkNextInRange:528 >> org.apache.commons.rng.core.source32.MersenneTwister: Too many failures for >> n = 10, sample size = 1000 (13 out of 500 tests failed, chi2 > >> 21.666=[23.920, 21.720, 22.440, 24.280, 23.920, 27.320, 22.680, 22.160, >> 34.440, 27.020, 29.680, 22.260, 22.400]) >> [INFO] Run 2: PASS >> [INFO] >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextLongInRange(UniformRandomProvider)[14] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextLongInRange:155->checkNextLongInRange:404->checkNextInRange:528 >> org.apache.commons.rng.core.source32.XoShiRo128Plus: Too many failures for >> n = 31, sample size = 1000 (12 out of 500 tests failed, chi2 > >> 21.666=[24.651, 24.155, 21.915, 21.778, 22.584, 21.667, 22.667, 24.093, >> 31.256, 25.519, 22.101, 21.750]) >> [INFO] Run 2: PASS >> [INFO] >> [WARNING] >> org.apache.commons.rng.core.ProvidersCommonParametricTest.testUniformNextLongInRange(UniformRandomProvider)[16] >> [ERROR] Run 1: >> ProvidersCommonParametricTest.testUniformNextLongInRange:156->checkNextLongInRange:404->checkNextInRange:528 >> org.apache.commons.rng.core.source32.PcgXshRr32: Too many failures for n = >> 32, sample size = 1000 (12 out of 500 tests failed, chi2 > 21.666=[24.699, >> 23.464, 25.043, 31.397, 31.739, 22.288, 24.637, 22.547, 22.483, 21.939, >> 22.795, 21.973]) >> [INFO] Run 2: PASS >> [INFO] >> [INFO] >> [WARNING] Tests run: 2058, Failures: 0, Errors: 0, Skipped: 6, Flakes: 7 >> >> Any thoughts? >> >>> On Jul 8, 2024, at 4:21 PM, Alex Herbert <alex.d.herb...@gmail.com> >> wrote: >>> >>> Hi Gary, >>> >>> Thanks for the feedback. I'll fix the docs module tomorrow. It is only >> for >>> an aggregate javadoc report for the site. It depends on all the other jar >>> modules and builds a single aggregate javadoc. This requires it to be a >> jar >>> artifact in the pom. But there is no output jar. I'll disable japicmp to >>> fix this. Really japicmp should handle a non-existent jar but it doesn't. >>> >>> The email I sent out is edited from the one built by the commons release >>> plugin. But I did not edit the site link (oops). The site is not staged >>> correctly for a multi-module jar. So I built the site separately and put >> it >>> in a personal Apache site. This makes it easy to see all the site >> including >>> the examples (which are not officially released but documented in the >> site). >>> >>> Note: I cancelled this vote as some intentionally private constructors >> were >>> public. >>> >>> Alex >>> >>> >>> On Mon, 8 Jul 2024 at 18:52, Gary D. Gregory <ggreg...@apache.org> >> wrote: >>> >>>> Oops, actually 'mvn clean install -DskipTests japicmp:cmp' fails with >>>> >>>> [INFO] Reactor Summary for Apache Commons RNG 1.6: >>>> [INFO] >>>> [INFO] Apache Commons RNG ................................. SUCCESS [ >>>> 21.207 s] >>>> [INFO] Apache Commons RNG Client API ...................... SUCCESS [ >>>> 24.159 s] >>>> [INFO] Apache Commons RNG Core ............................ SUCCESS [ >>>> 25.444 s] >>>> [INFO] Apache Commons RNG Simple .......................... SUCCESS [ >>>> 20.091 s] >>>> [INFO] Apache Commons RNG Sampling ........................ SUCCESS [ >>>> 42.508 s] >>>> [INFO] Apache Commons RNG (Bill of Materials) ............. SUCCESS [ >>>> 3.610 s] >>>> [INFO] Apache Commons RNG Documentation ................... FAILURE [ >>>> 3.601 s] >>>> [INFO] >>>> ------------------------------------------------------------------------ >>>> [INFO] BUILD FAILURE >>>> [INFO] >>>> ------------------------------------------------------------------------ >>>> [INFO] Total time: 02:22 min >>>> [INFO] Finished at: 2024-07-08T13:42:52-04:00 >>>> [INFO] >>>> ------------------------------------------------------------------------ >>>> [ERROR] Failed to execute goal >>>> com.github.siom79.japicmp:japicmp-maven-plugin:0.21.2:cmp (default-cli) >> on >>>> project commons-rng-docs: The following artifacts could not be resolved: >>>> org.apache.commons:commons-rng-docs:jar:1.5 (absent): Could not find >>>> artifact org.apache.commons:commons-rng-docs:jar:1.5 in central ( >>>> https://repo.maven.apache.org/maven2) -> [Help 1] >>>> >>>> The POM for that module is probably missing some configuration. >>>> >>>> Gary >>>> >>>> On 2024/07/08 17:50:41 "Gary D. Gregory" wrote: >>>>> Hello, >>>>> >>>>> +1 >>>>> >>>>> This email contains this link that returns a 404 for >>>>> >>>>> >>>> >> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/site/changes-report.html >>>>> >>>>> I see that for some reason, the site is in a personal folder instead of >>>> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/site, so I >>>> imagine this is some search and replace error. >>>>> >>>>> ASC OK. >>>>> SHA512 OK. >>>>> Running 'mvn' OK. >>>>> Running 'mvn clean install -DskipTests japicmp:cmp' OK. >>>>> >>>>> Using: >>>>> >>>>> Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256) >>>>> Maven home: C:\java\apache-maven-3.9.8 >>>>> Java version: 17.0.11, vendor: Eclipse Adoptium, runtime: C:\Program >>>> Files\Eclipse Adoptium\jdk-17.0.11.9-hotspot >>>>> Default locale: en_US, platform encoding: Cp1252 >>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: >> "windows" >>>>> >>>>> Gary >>>>> >>>>> On 2024/07/05 12:21:37 Alex Herbert wrote: >>>>>> We have fixed a few bugs and added enhancements since Apache Commons >>>> RNG >>>>>> 1.5 was released, so I would like to release Apache Commons RNG 1.6. >>>>>> >>>>>> Apache Commons RNG 1.6 RC1 is available for review here: >>>>>> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1 (svn >>>>>> revision 70143) >>>>>> >>>>>> The Git tag commons-rng-1.6-RC1 commit for this RC is >>>> commons-rng-1.6-RC1 >>>>>> which you can browse here: >>>>>> >>>>>> >>>> >> https://gitbox.apache.org/repos/asf?p=commons-rng.git;a=commit;h=commons-rng-1.6-RC1 >>>>>> You may checkout this tag using: >>>>>> git clone https://gitbox.apache.org/repos/asf/commons-rng.git >>>> --branch >>>>>> commons-rng-1.6-RC1 commons-rng-1.6-RC1 >>>>>> >>>>>> Maven artifacts are here: >>>>>> >>>>>> >>>> >> https://repository.apache.org/content/repositories/orgapachecommons-/org/apache/commons/commons-rng/1.6/ >>>>>> >>>>>> These are the artifacts and their hashes: >>>>>> >>>>>> #Release SHA-512s >>>>>> #Fri Jul 05 12:42:04 BST 2024 >>>>>> >>>> >> commons-rng-1.6-bin.tar.gz=bd6475337043085fe48e0da4a10bbc06157b0a5cf0496e9784aedcdd27e60141575e127a49ae6dab21f2ec404e3f9b834c56155abfe58b0d8c7761f543bffdb8 >>>>>> >>>> >> commons-rng-1.6-bin.zip=dbbc2fe27c024302dfce1d62414aa3718a8bfcdfd639e6acfae5828f624ea4bbcd01511e16b429be6873e8952354eb542f90b6ff62f3a00a5e55af9a5d7480bf >>>>>> >>>> >> commons-rng-1.6-src.tar.gz=6c1f89aaf3889296d830bcfd3f05ff650670ca3b8aa5b3c395b59801615f4b98e076f6731c6c7922adc348fa5f0367e1dec0bcb176318a589b61961b064666c1 >>>>>> >>>> >> commons-rng-1.6-src.zip=af96ba992540583e515a642a61ab03b98201f8f8e4b3ce7512f497feadf19aeea8b4fab2a8e113b7b65021744212f2b357a3907e2e1df74b4ea253875123d689 >>>>>> >>>> >> org.apache.commons_commons-rng-1.6.spdx.json=9742c043487e1e49fa2dd824370bef005148af757c0f8626188edf4d39deafe115e4caa10b015892e9d32c1fe1b911d4b4119cd33e59d5107900d8c1ad145112 >>>>>> >>>>>> Signatures may be validated on a system supporting a bash Unix shell >> by >>>>>> executing: >>>>>> svn co https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/ >>>>>> cd 1.6-RC1 >>>>>> chmod +x ./signature-validator.sh >>>>>> for m in client-api core simple sampling bom; do >>>>>> ./signature-validator.sh >>>>>> >>>> >> https://repository.apache.org/content/repositories/orgapachecommons-1754/org/apache/commons/commons-rng-${m}/1.6/ >>>> ; >>>>>> done >>>>>> >>>>>> The source code contains examples that are not part of the public API. >>>>>> These examples contain Java 11 modules and are enabled using a profile >>>> (see >>>>>> below). >>>>>> >>>>>> Note: Testing randomness using statistical thresholds results in >>>> failures >>>>>> at a given probability. The 'maven-surefire-plugin' is configured to >>>> re-run >>>>>> tests that fail, and pass the build if they succeed within the >> allotted >>>>>> number of reruns (the test will be marked as 'flaky' in the report). >>>>>> >>>>>> I have tested this with 'mvn clean install' and 'mvn clean package >> site >>>>>> site:stage -Pcommons-rng-examples' using: >>>>>> >>>>>> Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) >>>>>> Maven home: /Users/ah403/software/apache-maven-3 >>>>>> Java version: 11.0.23, vendor: Eclipse Adoptium, runtime: >>>>>> /Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home >>>>>> Default locale: en_GB, platform encoding: UTF-8 >>>>>> OS name: "mac os x", version: "14.5", arch: "aarch64", family: "mac" >>>>>> >>>>>> Details of changes since 1.5 are in the release notes: >>>>>> >>>>>> >>>> >> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/RELEASE-NOTES.txt >>>>>> >>>>>> >>>> >> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/site/changes-report.html >>>>>> >>>>>> Site: >>>>>> >>>> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/index.html >>>>>> (note some *relative* links are broken and the 1.6 directories are >>>> not >>>>>> yet created - these will be OK once the site is deployed.) >>>>>> >>>>>> JApiCmp Report (compared to 1.5): >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-client-api/japicmp.html >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-core/japicmp.html >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-simple/japicmp.html >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-sampling/japicmp.html >>>>>> >>>>>> RevApi Report (compared to 1.5): >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-client-api/revapi-report.html >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-core/revapi-report.html >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-simple/revapi-report.html >>>>>> >>>>>> >>>> >> https://home.apache.org/~aherbert/commons-rng-1.6-RC1-site/commons-rng-sampling/revapi-report.html >>>>>> >>>>>> RAT Report: >>>>>> >>>>>> >>>> >> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/site/rat-report.html >>>>>> >>>>>> KEYS: >>>>>> https://downloads.apache.org/commons/KEYS >>>>>> >>>>>> Please review the release candidate and vote. >>>>>> This vote will close no sooner than 72 hours from now. >>>>>> >>>>>> [ ] +1 Release these artifacts >>>>>> [ ] +0 OK, but... >>>>>> [ ] -0 OK, but really should fix... >>>>>> [ ] -1 I oppose this release because... >>>>>> >>>>>> Thank you, >>>>>> >>>>>> Alex Herbert, >>>>>> Release Manager (using key BC87A3FD0A54480F0BADBEBD21939FF0CA2A6567) >>>>>> >>>>>> The following is intended as a helper and refresher for reviewers. >>>>>> >>>>>> Validating a release candidate >>>>>> ============================== >>>>>> >>>>>> These guidelines are NOT complete. >>>>>> >>>>>> Requirements: Git, Java, Maven. >>>>>> >>>>>> You can validate a release from a release candidate (RC) tag as >>>> follows. >>>>>> >>>>>> 1a) Clone and checkout the RC tag >>>>>> >>>>>> git clone https://gitbox.apache.org/repos/asf/commons-rng.git >> --branch >>>>>> commons-rng-1.6-RC1 commons-rng-1.6-RC1 >>>>>> cd commons-rng-1.6-RC1 >>>>>> >>>>>> 1b) Download and unpack the source archive from: >>>>>> >>>>>> https://dist.apache.org/repos/dist/dev/commons/rng/1.6-RC1/source >>>>>> >>>>>> 2) Check Apache licenses >>>>>> >>>>>> This step is not required if the site includes a RAT report page which >>>> you >>>>>> then must check. >>>>>> >>>>>> mvn apache-rat:check >>>>>> >>>>>> 3) Check binary compatibility >>>>>> >>>>>> Newer components use JApiCmp with the japicmp Maven Profile: >>>>>> >>>>>> This step is not required if the site includes a JApiCmp report page >>>> which >>>>>> you then must check. >>>>>> >>>>>> mvn install -DskipTests -P japicmp japicmp:cmp >>>>>> >>>>>> 4) Build the package >>>>>> >>>>>> mvn -V clean package >>>>>> >>>>>> You can record the Maven and Java version produced by -V in your VOTE >>>> reply. >>>>>> To gather OS information from a command line: >>>>>> Windows: ver >>>>>> Linux: uname -a >>>>>> >>>>>> 5) Build the site for a multi-module project >>>>>> >>>>>> mvn site >>>>>> mvn site:stage >>>>>> Check the site reports in: >>>>>> - Windows: target\site\index.html >>>>>> - Linux: target/site/index.html >>>>>> >>>>>> Note that the project reports are created for each module. >>>>>> Modules can be accessed using the 'Project Modules' link under >>>>>> the 'Project Information' menu (see <path-to-site>/modules.html). >>>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>> >>>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >> For additional commands, e-mail: dev-h...@commons.apache.org >> >>
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
