Md-Arif-Hasan opened a new pull request, #26511:
URL: https://github.com/apache/flink/pull/26511
# Add GitHub Actions workflow to verify flaky test
## Purpose
This PR adds a custom GitHub Actions workflow named `verify-cep-flaky.yml`
to confirm that the test
`org.apache.flink.cep.operator.CEPOperatorTest.testCEPOperatorCleanupEventTime`
is flaky.
The test passes under normal execution but fails when executed with NonDex,
confirming the presence of non-deterministic behavior.
## Summary of Changes
- Added a new workflow: `.github/workflows/verify-cep-flaky.yml`
- The workflow:
- Sets up **JDK 8** using `actions/setup-java`
- Checks out the specific commit where the flaky test was identified
- Builds the project using Maven with tests skipped
- Runs the test normally (expected to pass)
- Runs the test again using **NonDex** (expected to fail)
- Verifies flakiness by comparing exit codes and logs
## Output
The GitHub Actions build shows:
- ✅ Normal test run passes
- ❌ NonDex test run fails
- Final verification logs include: Test is flaky: Passes without NonDex but
fails with NonDex
## Logs and Output
Logs and Output:
You can find the logs and output in the attached build artifacts, or view
the successful build at:
(https://github.com/Md-Arif-Hasan/flink/actions/runs/14682914330)
- **CI Build Log**:
[ci-build.log](https://drive.google.com/file/d/1rPmbMy5EFj_J9pz9fKmKcAJApktrn9Lv/view?usp=sharing)
- **Workflow YAML**:
[verify-cep-flaky.yml](https://github.com/Md-Arif-Hasan/flink/blob/verify-cep-flaky-test/.github/workflows/verify-cep-flaky.yml)
## Notes
This workflow automates the documentation of the flakiness of
`CEPOperatorTest.testCEPOperatorCleanupEventTime` using GitHub Actions.
The CI logs confirm that the test:
- Passes under normal conditions.
- Fails when executed with **NonDex**, exposing non-deterministic behavior.
The flakiness arises from assumptions about deterministic behavior in Java
collections, which NonDex breaks by randomizing collection iteration order.
By using this workflow, flaky tests can be **detected** and **verified** in
a controlled and automated manner, helping to improve the stability of the
project's test suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
