goutamadwant opened a new pull request, #12852:
URL: https://github.com/apache/seatunnel/pull/12852
### Purpose of this pull request
Fixes #12851.
`FlussSinkIT` sometimes hangs until `all-connectors-it-6` hits its 2h30m
timeout. The `fluss/fluss:0.7.0` test server can miss a table creation
(apache/fluss#1018, fixed in Fluss 1.0): the table is registered, but its
bucket never gets a leader, and the SeaTunnel job waits for one forever.
This PR changes `FlussTestBase` only:
- after creating a table, wait up to 30 s until every bucket has a leader
(`listOffsets` fails while one is missing); if not, drop and recreate the
table, up to 3 attempts, then fail with the last error;
- add a 15-minute `@Timeout` to each Fluss test case (the slowest case waits
up to about 8 minutes), so any other hang fails one case instead of the whole
job.
**Before:** a lost table blocks the job until the 2h30m job timeout.
**After:** the table is recreated within about 30 s; if a case still hangs,
it fails after 15 minutes and the remaining cases run.
### Does this PR introduce _any_ user-facing change?
No. E2E tests only.
### How was this patch tested?
- Created the state the bug leaves behind (table and assignment registered
in ZooKeeper, never processed by the coordinator):
- the new leader check times out after 30 s, and a client write to the
table blocks;
- `createTable` from this PR recreates it in 3 s; the new table passes the
check and accepts the write.
- Ran a SeaTunnel job against such a table on Flink 1.18, Flink 1.20, Spark
3.3 and Zeta, with the timeout set to 3 minutes: the job never returned on any
engine, each case failed at 3 minutes, and the next case and teardown still ran.
- `FlussSinkIT` and `FlussSourceIT` on Flink 1.18, Flink 1.20, Spark 3.3 and
Zeta: 18/18 pass on JDK 11 and 17.
- The race itself did not reproduce locally in 1,860 table creates (it shows
up in about 4% of it-6 runs in CI).
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
