Or alternatively, if using random values each time, have it retry the test with a different value. It's typically better to use an actual property testing library for these types of tests anyways. One example library I found is https://jqwik.net/ (these types of testing libraries are more common in functional programming like in Scala).
On Thu, 6 Aug 2020 at 09:59, Matt Sicker <boa...@gmail.com> wrote: > > Choose a seed value for the `new Random()` constructor and the tests > will be deterministic. > > On Thu, 6 Aug 2020 at 09:57, Rob Tompkins <chtom...@gmail.com> wrote: > > > > > > > > > On Aug 6, 2020, at 10:56 AM, Gary Gregory <garydgreg...@gmail.com> wrote: > > > > > > This is all fine and good but how would you fix the test such that it does > > > not fail randomly. PR anyone? > > > > Either static inputs for determinism, or putting a probabilistic boundary > > in which the solution can fall. > > > > -Rob > > > > > > > > Gary > > > > > > On Thu, Aug 6, 2020 at 10:54 AM Matt Sicker <boa...@gmail.com> wrote: > > > > > >> The ECC stuff I mostly learned about from various Bernstein papers > > >> like this one: https://cr.yp.to/newelliptic/nistecc-20160106.pdf > > >> > > >> On Thu, 6 Aug 2020 at 09:50, Rob Tompkins <chtom...@gmail.com> wrote: > > >>> > > >>> > > >>> > > >>>> On Aug 6, 2020, at 10:42 AM, Matt Sicker <boa...@gmail.com> wrote: > > >>>> > > >>>> Well, for testing RNGs, I can understand using property testing, yes. > > >>>> It would also be useful for testing fuzzing scenarios like making sure > > >>>> the GCM tag is invalid for any random input data (giving a near zero > > >>>> probability of valid data) or that an elliptic curve implementation > > >>>> doesn't leak out information about points outside the curve or respond > > >>>> to invalid inputs improperly or things like that. > > >>> > > >>> +1 - the elliptic curve stuff I’ll have to defer to you on as I’m less a > > >> number theorist and more of a logician. > > >>> > > >>> -Rob > > >>> > > >>>> > > >>>> On Thu, 6 Aug 2020 at 09:37, Rob Tompkins <chtom...@gmail.com> wrote: > > >>>>> > > >>>>> > > >>>>> > > >>>>>> On Aug 6, 2020, at 10:33 AM, Matt Sicker <boa...@gmail.com> wrote: > > >>>>>> > > >>>>>> Now I hope we don't have unit tests depending on non-static state for > > >>>>>> its random number generator! ;) > > >>>>> > > >>>>> We actually do have a considerable number of those in our projects > > >> where we use probabilistic epsilons on the output. See commons-rng. Note, > > >> Gilles is quite good at writing such tests. > > >>>>> > > >>>>> -Rob > > >>>>> > > >>>>>> I'd expect a crypto library's test > > >>>>>> suites to include several hard-coded known-good and known-bad > > >>>>>> ciphertexts with static keys/IVs similar to the test cases presented > > >>>>>> in their RFCs (especially since said tests are typically small enough > > >>>>>> to copy/paste the binary data fairly easily). > > >>>>>> > > >>>>>> On Thu, 6 Aug 2020 at 08:19, Gary Gregory <garydgreg...@gmail.com> > > >> wrote: > > >>>>>>> > > >>>>>>> On Thu, Aug 6, 2020 at 8:31 AM Alex Remily <alex.rem...@gmail.com> > > >> wrote: > > >>>>>>> > > >>>>>>>> No problem. I'll do it when I get home tonight. > > >>>>>>>> > > >>>>>>> > > >>>>>>> Thanks Alex! > > >>>>>>> > > >>>>>>> Gary > > >>>>>>> > > >>>>>>> > > >>>>>>>> > > >>>>>>>> On Thu, Aug 6, 2020, 8:25 AM Gary Gregory <garydgreg...@gmail.com> > > >> wrote: > > >>>>>>>> > > >>>>>>>>> Hi Alex, > > >>>>>>>>> > > >>>>>>>>> Would you mind creating that ticket with that info? > > >>>>>>>>> > > >>>>>>>>> Thank you, > > >>>>>>>>> Gary > > >>>>>>>>> > > >>>>>>>>> On Thu, Aug 6, 2020, 08:10 Alex Remily <alex.rem...@gmail.com> > > >> wrote: > > >>>>>>>>> > > >>>>>>>>>> That is an intermittent issue that I haven't been able to > > >> reliably > > >>>>>>>>>> reproduce. As I recall, the test that's failing is supposed to > > >> fail, > > >>>>>>>> but > > >>>>>>>>>> in a different way. I think it's supposed to fail because of a > > >> short > > >>>>>>>>>> buffer but occasionally fails because of an internal error, and > > >> when > > >>>>>>>> that > > >>>>>>>>>> happens this test fails. I don't know when it was introduced. > > >> We > > >>>>>>>> should > > >>>>>>>>>> probably document it in jira and or realese notes. > > >>>>>>>>>> > > >>>>>>>>>> On Wed, Aug 5, 2020, 10:53 PM Gary Gregory < > > >> garydgreg...@gmail.com> > > >>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Hi All: > > >>>>>>>>>>> > > >>>>>>>>>>> I am seeing what may be a random AEADBadTagException in > > >>>>>>>> GcmCipherTest? > > >>>>>>>>>>> > > >>>>>>>>>>> For example: > > >>>>>>>>>>> > > >>>>>>>>>>> [ERROR] > > >>>>>>>>>> > > >> testGcmTamperedData(org.apache.commons.crypto.cipher.GcmCipherTest) > > >>>>>>>>>>> Time elapsed: 0.015 s <<< ERROR! > > >>>>>>>>>>> 881java.lang.Exception: Unexpected exception, > > >>>>>>>>>>> expected<javax.crypto.AEADBadTagException> but > > >>>>>>>>>>> was<java.lang.InternalError> > > >>>>>>>>>>> 882 at > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >> org.apache.commons.crypto.cipher.GcmCipherTest.testGcmTamperedData(GcmCipherTest.java:224) > > >>>>>>>>>>> 883 > > >>>>>>>>>>> 884 > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> Any thoughts? > > >>>>>>>>>>> > > >>>>>>>>>>> The above is from > > >>>>>>>>>>> > > >> https://travis-ci.org/github/apache/commons-crypto/jobs/715348986 > > >>>>>>>>>>> > > >>>>>>>>>>> Gary > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> Matt Sicker <boa...@gmail.com> > > >>>>>> > > >>>>>> --------------------------------------------------------------------- > > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > >>>>>> For additional commands, e-mail: dev-h...@commons.apache.org > > >>>>>> > > >>>>> > > >>>>> > > >>>>> --------------------------------------------------------------------- > > >>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > >>>>> For additional commands, e-mail: dev-h...@commons.apache.org > > >>>>> > > >>>> > > >>>> > > >>>> -- > > >>>> Matt Sicker <boa...@gmail.com> > > >>>> > > >>>> --------------------------------------------------------------------- > > >>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > >>>> For additional commands, e-mail: dev-h...@commons.apache.org > > >>>> > > >>> > > >>> > > >>> --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > >>> For additional commands, e-mail: dev-h...@commons.apache.org > > >>> > > >> > > >> > > >> -- > > >> Matt Sicker <boa...@gmail.com> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > >> For additional commands, e-mail: dev-h...@commons.apache.org > > >> > > >> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > -- > Matt Sicker <boa...@gmail.com> -- Matt Sicker <boa...@gmail.com> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
