On Tuesday, June 23, 2020 at 3:37:59 AM UTC+10, Nils Bruin wrote: > > On Monday, June 22, 2020 at 6:18:11 AM UTC-7, Michael Orlitzky wrote: >> >> Doing so won't consume any extra time >> on any individual machine, and the multitudes of reviewers and patchbots >> running the test on different examples will ferret out any corner cases. >> >> One of the concerns is that you won't get very good error reports this > way: without good condition reporting, you might get reports "sometimes > this test fails" and in many cases people will probably not bother > reporting because the problem went away the next time. In my experience, > "corner cases" in mathematical algorithms are often NOT found with random > input, because the probability measure of their triggering inputs is > vanishingly small. > > A cheap compromise might be to make the "starting seed" for tests > configurable. The default would be just the seed we have now, but if people > want to set it to another value, they can go ahead. It could be helpful if > the value used is part of the test result report. If people want, they can > then just script the testing with variable seeds. It would offer a cheap > workaround to get a bit more coverage in your tests (at some point) if you > think your code is particularly sensitive to output from the random > generator. >
I agree with Nils about this, and I like his proposal. I also think the random number might (and should) change if other doctests are changed around it. However, having something that consistently fails is better than something that does only part of the time. I have had doctests failing in a file run as a standalone than within a folder due to a state being cached in a separate file. There are also some transient errors that occur when running the entire library tests. So having randomness in the doctests makes it harder to detect if these are actual bugs or not (in my previous example, it was not a bug in the code but in the doctest setup). Best, Travis -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/928b6da8-f9ab-49ac-a9a5-15455f467724o%40googlegroups.com.
