Hi Team, Recently I have seen plenty of flaky tests again in the pre-commit runs but only seen Zoltan's commits to handle these tests.
Some time ago Zoltan Haindrich created a nice tool to check for flaky tests for everyone: - http://ci.hive.apache.org/job/hive-flaky-check/ <http://ci.hive.apache.org/job/hive-flaky-check/> Everyone can use the jenkins job above to run a specific test - or set of tests - 100 times, so we can check flakiness on the master branch. Some example parametrization: Testing a whole class: -pl ql -Dtest=TestWorker Testing multiple classes: -pl itests/hive-unit -Dtest=TestStatsReplicationScenariosACID,TestStatsReplicationSce​nariosACIDNoAutogather Running a qtest: -pl itests/qtest -Dtest=TestMiniLlapLocalCliDriver -Dqfile=replication_metrics_ingest.q Running a specific test method: -pl itests/hive-unit -Dtest=TestJdbcWithMiniHS2#testHttpRetryOnServerIdleTimeout If I remember correctly we agreed to the following process to keep the pre-commit runs green: Run the flaky test jenkins job to validate that the test is indeed flaky Create a jira for fixing the flaky test (if you know someone interested, please ping them). Good example: https://issues.apache.org/jira/browse/HIVE-25288 <https://issues.apache.org/jira/browse/HIVE-25288> Create the PR for disabling the test (please disable only the test method if possible). Good example: https://github.com/apache/hive/commit/865664b <https://github.com/apache/hive/commit/865664b> If the flakiness is confirmed, push the change immediately (no need to have a review or green test run) If you can not push the PR yourself, please ping a committer It would be good if everyone could follow the process above and would help us keep the pre-commit test runs green, so the work would not depend on a single member of our team. Thanks, Peter
