SEZ9 commented on issue #12339: URL: https://github.com/apache/seatunnel/issues/12339#issuecomment-5755238302
Thanks for the detailed breakdown, that clarifies where the load actually lands. Since the barrier trigger and ACK handling run on the coordinator-service pool, which is already tunable via `engine.coordinator-service.core-thread-num` / `max-thread-num`, I agree that adding a second checkpoint-specific dial without a signal to tune against is not ideal. A few follow-ups: 1. The `thenAccept` path that can run the blocking `CompletableFuture.allOf(completableFutureArray).get()` inline on a dispatch thread sounds like it should be closed as part of this change. Will the `thenAcceptAsync(..., executorService)` fix go into the candidate PR? 2. If the option does end up in this PR, your compromise of a single `checkpoint.scheduler-dispatch-thread-num` with default `0` meaning auto works for me, since existing deployments keep the same behavior. Let's see what the dev list says. 3. Metrics (dispatch queue depth, active dispatch threads, scheduling delay) as a follow-up sounds good. On your question about a deployment with drifting triggers: my concern comes from the separated-deployment scenario in general rather than a specific measured case, so I don't have a reproduction to offer. Could you also post the dev@ `[DISCUSS]` thread link here once it is sent, so the decision on the option is recorded alongside this issue? <!-- streview-comment:1217 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
