The chi-square critical value (13.82) is correct: >>> from scipy.stats import chi2 >>> chi2(2).isf(0.001) 13.815510557964274
The test seems to fail with the expected frequency when run locally. I annotated with: @RepeatedTest(value = 100000) I observe 93 failures (just under 1 in 1000). So it is strange this fails a lot on the GH CI build. We could just use a fixed Random argument to the call that is ultimately performing the random string generation: random(count, 0, chars.length, false, false, chars, random()); Switch the test to: Random rng = new Random(0xdeadbeef) gen = RandomStringUtils.random(6, 0, 3, false, false, chars, rng); You will see a drop in coverage by not exercising the public API. The alternative is to change the chi-square critical value: 1 in 10,000: 18.420680743952364 1 in 100,000: 23.025850929940457 1 in 1,000,000: 27.631021115928547 Alex On Fri, 20 Oct 2023 at 18:44, Elliotte Rusty Harold <elh...@ibiblio.org> wrote: > > It's possible the chi square test is miscalculated. Perhaps some stats > expert can check that. It's also possible the chi square test isn't > the right one to use here. Again, consult a stats expert. > > It's also very possible that the randomness is not nearly as random as > it's supposed to be. That's incredibly common, and that might be > noticeable given the very short three-letter character set [a, b, c] > being picked from. > > On Fri, Oct 20, 2023 at 1:31 PM Gary D. Gregory <ggreg...@apache.org> wrote: > > > > Despite the failure comment: > > > > RandomStringUtilsTest.testRandomStringUtilsHomog:474 test homogeneity -- > > will fail about 1 in 1000 times ==> expected: <true> but was: <false> > > > > This test fails a LOT more than once every 1000 times, based on how many > > GitHub builds I need to restart every week. > > > > What can be done to make this test more resilient? > > > > TY! > > Gary > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
