fhan688 opened a new pull request, #20271:
URL: https://github.com/apache/hudi/pull/20271
### Describe the issue this Pull Request addresses
`ConnectTransactionCoordinator.stop()` does not reliably terminate the
coordinator event loop.
The event loop currently runs with `while (true)`. Although `stop()`
updates `hasStarted` and eventually interrupts the executor thread, the
`InterruptedException` raised by `events.poll()` is only logged and the loop
continues running.
As a result:
- The coordinator executor thread may remain alive after `stop()` returns.
- Repeated coordinator creation can accumulate leaked threads.
- Pending events may still be processed after the coordinator has been
stopped.
- Events submitted concurrently with shutdown may throw
`RejectedExecutionException`.
### Summary and Changelog
This PR fixes the lifecycle management of the Kafka Connect transaction
coordinator.
Changes include:
- Make the coordinator event loop run only while the coordinator is active.
- Exit the event loop when the executor thread is interrupted.
- Restore the thread interrupt status before terminating the loop.
- Make `stop()` idempotent using an atomic state transition.
- Ensure executor cleanup happens even if coordinator deregistration fails.
- Cancel scheduled events and clear pending coordinator events during
shutdown.
- Interrupt the event-loop executor and wait for bounded termination.
- Ignore events submitted after shutdown.
- Handle the race between event submission and scheduler shutdown.
- Add a lifecycle test covering:
- Event-loop termination.
- Repeated calls to `stop()`.
- Event submission after shutdown.
- Coordinator deregistration occurring exactly once.
No code was copied from external sources.
### Impact
This change only affects the shutdown lifecycle of the Kafka Connect
transaction coordinator.
There are no changes to:
- Public APIs.
- Configuration options.
- Storage format.
- Existing transaction coordination behavior while the coordinator is
running.
After shutdown begins, queued or newly submitted coordinator events are
discarded. `stop()` performs a bounded wait of up to five seconds for the
event-loop executor to terminate.
### Risk Level
Low.
The change is localized to `ConnectTransactionCoordinator` lifecycle
handling. Existing coordinator tests and the new lifecycle regression test pass
successfully.
Validation performed:
```shell
mvn -o -Pspark3.5,flink2.2 \
-pl hudi-kafka-connect \
-DskipITs=true \
-Dtest=TestConnectTransactionCoordinator,TestConnectTransactionCoordinatorLifecycle
\
test
Test result:
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS
```
The module also passes Checkstyle and Apache RAT validation.
### Documentation Update
None. This PR does not introduce or change any public API, configuration,
or user-facing feature.
### Contributor's checklist
- [ ] Read through contributor's guide
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
