SEZ9 opened a new issue, #12788: URL: https://github.com/apache/seatunnel/issues/12788
### Search before asking - [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ### Description `CheckpointMonitorService` is constructed with a hardcoded history size in `SeaTunnelServer.startMaster()`: ```java checkpointMonitorService = new CheckpointMonitorService(engineContext, 32); ``` That 32 is how many completed checkpoints the monitor keeps **per pipeline** for the `/jobs/checkpoints/{jobId}` endpoint (`PipelineCheckpointOverview.addHistory` trims to it). There is no configuration option for it, so an operator cannot change the retained window. ### Why it matters The window bounds how far back a client can compare a job against its own past, which is what diagnosing a *gradual* slowdown needs — the case where a job is still `RUNNING`, both row counters are moving, `restoreCount` is 0, and the only evidence that anything changed is that checkpoints now take longer than they used to. How much history 32 entries buys depends entirely on the checkpoint interval, and the mismatch is large: | `checkpoint.interval` | history covered by 32 entries | |---|---| | 300000 ms (engine default) | ~2.5 hours | | 60000 ms | ~32 minutes | | 10000 ms (common for CDC) | **~5 minutes** | Five minutes of baseline is not enough to distinguish a trend from noise, and a short interval is exactly what a latency-sensitive CDC job is configured with. The trend rules added in #12688 consume this endpoint and are limited by it; the `seatunnel-cli` README has to tell users the window is 32 checkpoints because there is nothing they can do about it. ### Proposal Add `seatunnel.engine.checkpoint.monitor-history-size`, defaulting to 32 so current behaviour is unchanged: ```yaml seatunnel: engine: checkpoint: monitor-history-size: 128 ``` The history is retained in memory per pipeline, so a larger window costs memory proportional to the number of running pipelines; that trade-off belongs in the documentation rather than in a hidden constant. ### Usage Scenario A CDC job with `checkpoint.interval = 10000` that degrades over an hour. With the current fixed window the retained history only reaches back about five minutes, so by the time someone looks, the comparison that would show the degradation has already rolled out of the window. Raising the size to 128 covers roughly 20 minutes at that interval. ### Related issues - #12688 — the checkpoint trend rules that read this endpoint ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
