On Tue, 3 Jan 2023 09:39:59 GMT, Aleksey Shipilev <sh...@openjdk.org> wrote:
> This should help to speed up tests significantly. Currently, if we run "make > test" with a subset of tests, JTReg would still read the entirety of test > root to report on tests that were not run. Even with current suite of tests > it gets expensive. If you add more tests to suite -- for example, mounting a > large test archive like pre-generated fuzzer tests -- it starts to take a lot > of time. > > The reasonable default for older jtreg is `-report:executed`. My own CI -- > that has millions of tests mounted in `test/` -- was running with > `JTREG="OPTIONS=-report:executed"` for years now. > [CODETOOLS-7903323](https://bugs.openjdk.org/browse/CODETOOLS-7903323) > provides a new reporting option, `-report:files`, which makes this whole > business easier. This requires new jtreg. > > Motivational improvements: > > > $ time CONF=linux-x86_64-server-release make run-test > TEST=gc/epsilon/TestHelloWorld.java > > # Default JDK tree > > # Baseline > real 0m9.086s > user 0m22.857s > sys 0m4.202s > > # Patched > real 0m6.406s ; +40% faster > user 0m17.156s > sys 0m3.193s > > # +100K fuzzer tests in tree > > # Baseline > real 3m1.997s > user 3m16.643s > sys 0m10.490s > > # Patched > real 0m8.919s ; 20x faster > user 0m17.904s > sys 0m4.860s > > > Additional testing: > - [x] Ad-hoc timing tests (see above) > - [x] Reproducer from > [CODETOOLS-7903331](https://bugs.openjdk.org/browse/CODETOOLS-7903331) > - [x] Eyeballing the test results on passing tests with REPEAT_COUNT > 1 > - [x] Eyeballing the test results on intermittently failing tests with > RETRY_COUNT > 1 Hello Aleksey, this looks good to me. Good to see the timing improvements on `make test`. Please update the copyright years on the files before integrating. I see that in the additional testing section you note, running tests with RETRY_COUNT. Is that intentional? Does retry count play a role in the report generation? ------------- Marked as reviewed by jpai (Reviewer). PR: https://git.openjdk.org/jdk/pull/11824
