On Fri, 20 Sep 2024 18:29:22 GMT, Andy Goryachev <ango...@openjdk.org> wrote:
>> Converting system tests to junit5. >> >> Please see migration notes: >> https://github.com/andy-goryachev-oracle/Test/blob/main/doc/Tests/JUnit5Migration.md >> >> ### Notes: >> >> I see shutdown timeout on linux, this will be addressed by >> [JDK-8340403](https://bugs.openjdk.org/browse/JDK-8340403) >> >> QPathTest > executionError FAILED >> org.opentest4j.AssertionFailedError: Exceeded timeout limit of 10000 msec >> at >> app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:39) >> at app//org.junit.jupiter.api.Assertions.fail(Assertions.java:134) >> at app//test.util.Util.runAndWait(Util.java:156) >> at app//test.util.Util.runAndWait(Util.java:127) >> at app//test.util.Util.shutdown(Util.java:365) >> at >> app//test.com.sun.marlin.QPathTest.teardownOnce(QPathTest.java:146) >> >> >> This test might fail intermittently (?) on linux only: >> >> SetSceneScalingTest > testShowAndSetScene() FAILED >> org.opentest4j.AssertionFailedError: expected: <true> but was: <false> >> at >> app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) >> at >> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:40) >> at >> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:35) >> at >> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:179) >> at >> app//test.robot.javafx.stage.SetSceneScalingTest$TestShowSetSceneApp.test(SetSceneScalingTest.java:129) >> at >> app//test.robot.javafx.stage.SetSceneScalingTest$TestApp.runTest(SetSceneScalingTest.java:89) >> at >> app//test.robot.javafx.stage.SetSceneScalingTest.testShowAndSetScene(SetSceneScalingTest.java:193) > > Andy Goryachev has updated the pull request with a new target base due to a > merge or a rebase. The incremental webrev excludes the unrelated changes > brought in by the merge/rebase. The pull request contains 24 additional > commits since the last revision: > > - Merge remote-tracking branch 'origin/master' into 8339510.junit5.system > - more fixes > - rotate test > - review comments > - unused imports > - fixes 4 > - fixes 3 > - fixes 2 > - fixes 1 > - part 12 last > - ... and 14 more: https://git.openjdk.org/jfx/compare/42f2d9d3...777620ef The latest change look correct and fix most of the failures where the entire test class was failing. I still see a discrepancy between number of tests run before / after and there are more failures now. On master, I get: 1773 tests : 56 failures : 495 ignored With the latest version of this PR, I get: 1326 tests : 101 failures: 78 ignored So we are now running _fewer_ tests than before: `1773-495=1278` vs `1326-78=1248`. As for the failures, I ran it twice and got 102 failures the first time and 101 the second time. So there is likely some failure that isn't due to test instability. ------------- PR Comment: https://git.openjdk.org/jfx/pull/1569#issuecomment-2364576531
