[
https://issues.apache.org/jira/browse/HDDS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17934636#comment-17934636
]
Peter Lee commented on HDDS-12574:
----------------------------------
I think this case is also covered. I did some tests on my local machine.
{code:java}
$ export TARGET_TEST_CLASS="org.apache.ozone.test.**"
$ hadoop-ozone/dev-support/checks/junit.sh
-Dtest="org.apache.ozone.test.**.java"
TARGET_TEST_CLASS: org.apache.ozone.test.**.java
Finding project for test class: org.apache.ozone.test.**.java
Searching for files matching base name: org.apache.ozone.test.
No files found with direct name search, trying path-based search
TEST_CLASS_PATH pattern: org\/apache\/ozone\/test\/.java
Found 29 matching test file(s)
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/GenericTestUtils.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/IntLambda.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/JacocoServer.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/LambdaTestUtils.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/Log4j1Capturer.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/Log4j2Capturer.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/MetricsAsserts.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/OzoneTestBase.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/SpyInputStream.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/SpyOutputStream.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/TestClock.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/TimedOutTestsListener.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/package-info.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/tag/Flaky.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/tag/Native.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/tag/Slow.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/tag/Unhealthy.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-hdds/test-utils/src/test/java/org/apache/ozone/test/tag/package-info.java
Found test in project: ./hadoop-hdds/test-utils
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/AclTests.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/ClusterForTests.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/ConfigAssumptions.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/FreonTests.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/HATests.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/NonHATests.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/TestFreon.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/TestOzoneIntegrationHA.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/TestOzoneIntegrationNonHA.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/TestOzoneNonHAWithNativeACL.java
Found test in project: ./hadoop-ozone/integration-test
TEST_FILE:
./hadoop-ozone/integration-test/src/test/java/org/apache/ozone/test/package-info.java
Found test in project: ./hadoop-ozone/integration-test
Setting project list to: -pl
./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-hdds/test-utils,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test,./hadoop-ozone/integration-test{code}
It seems we need to make the project list unique.
> Speed up flaky-test-check workflow in CI
> ----------------------------------------
>
> Key: HDDS-12574
> URL: https://issues.apache.org/jira/browse/HDDS-12574
> Project: Apache Ozone
> Issue Type: Improvement
> Components: CI
> Reporter: Peter Lee
> Assignee: Peter Lee
> Priority: Major
> Labels: pull-request-available
>
> Since flaky-test-check workflow run with designated TestClass, so we can find
> which package the TestClass belongs to then use -pl args to only build
> necessary packages
>
> I made a POC which shows great improvement in test running time:
> Test with TestDeadNodeHandler
> * run with master CI file:
> [https://github.com/peterxcli/ozone/actions/runs/13817211733] (takes 19mins
> * run with speedup-flaky-test-check-in-ci POC branch:
> [https://github.com/peterxcli/ozone/actions/runs/13817907647] (takes 8mins
>
> This POC also support duplicated class name across different packages, and
> the wildcard matching
>
> POC git diff:
> [https://github.com/apache/ozone/compare/master...peterxcli:ozone:speedup-flaky-test-check-in-ci?expand=1]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
