There's actually quite a lot of docs related to Lucene tests (my remark was meant at the randomizedtesting package) - see here: https://github.com/apache/lucene/blob/main/help/tests.txt
The tests.timeoutSuite parameter could be added/ explained there too. I'm not sure how much it's needed though - it's the first time anybody ever mentioned it. :) D. On Thu, Feb 15, 2024 at 8:55 PM Shubham Chaudhary <shubhmas...@gmail.com> wrote: > I think this information could sit well within the dev-docs in lucene i.e. > "randomized testing in lucene". This would make it more discoverable as > well and there is already a lack of proper docs as Dawid pointed?. We could > add some references to docs like randomized testing core concepts > <https://github.com/randomizedtesting/randomizedtesting/wiki/Core-Concepts>. > If this idea makes sense I could try to write some doc around it and open a > PR. I would like to know your thoughts on this? Thanks! > > - Shubham > > On Thu, Feb 15, 2024 at 10:23 PM Dawid Weiss <dawid.we...@gmail.com> > wrote: > >> >> Sorry, the docs are not the best, I know. >> >> It's documented here - >> >> https://github.com/randomizedtesting/randomizedtesting/blob/master/randomized-runner/src/main/java/com/carrotsearch/randomizedtesting/SysGlobals.java#L186-L197 >> >> So: >> >> 1) if you pass tests.timeoutSuite=1000 this changes the default value for >> all classes that don't define any explicit timeout using an annotation; >> classes that do have an annotation, >> use the annotation's value, >> 2) if you pass tests.timeoutSuite=1000! then this overrides everything - >> the default value and all annotations. >> >> I vaguely recall option (2) was added specifically for nightlies which >> bumped the iteration multiplier - this affected tests that normally ran >> fairly fast >> but during nightly runs could run slower than anticipated. >> >> D. >> >> >> On Thu, Feb 15, 2024 at 3:18 PM David Smiley <dsmi...@apache.org> wrote: >> >>> Oh; I didn't know that took precedence -- makes sense. Hopefully a >>> test subclass (like SolrTestCase) could override it as well. >>> >>> On Mon, Feb 12, 2024 at 2:09 PM Dawid Weiss <dawid.we...@gmail.com> >>> wrote: >>> > >>> > >>> > You can override the defaults using sysprops in your CI builds - >>> > >>> > -Ptests.timeoutSuite=1000! >>> > >>> > takes precedence over any annotations (1 second). >>> > >>> > Dawid >>> > >>> > On Mon, Feb 12, 2024 at 7:53 PM David Smiley <dsmi...@apache.org> >>> wrote: >>> >> >>> >> Looking at LuceneTestCase, I see the annotation from >>> RandomizedTesting: >>> >> @TimeoutSuite(millis = 2 * TimeUnits.HOUR) >>> >> This matches my observations of some builds that timed out, perhaps >>> >> some flaky test hanging in Solr (that extends LuceneTestCase). >>> >> Looking at this annotation, there is further documentation that the >>> >> default can be set via sysprop tests.timeoutSuite. Wouldn't doing >>> >> that make more sense than hard-coding this figure in LuceneTestCase? >>> >> For example, I'd like to have a normal/default test run have a low >>> >> timeout (10min?) but on a "nightly" run on CI, use much higher. Not 2 >>> >> hours though; individual tests needing so much should have a >>> >> TimeoutSuite applied to them. >>> >> >>> >> ~ David Smiley >>> >> Apache Lucene/Solr Search Developer >>> >> http://www.linkedin.com/in/davidwsmiley >>> >> >>> >> --------------------------------------------------------------------- >>> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>>
