chestnut-c commented on PR #9552: URL: https://github.com/apache/seatunnel/pull/9552#issuecomment-3056777354
> > why checkpoint will be disabled, and the system will still create checkpoint Storage later. > > When we disabled checkpoints, it is indeed unnecessary to create extra checkpoint storage. But re-examining this code, we have actually created the checkpoint storage when the server starts. > > https://github.com/apache/seatunnel/blob/3836c97a62079f1e8d7ca7a9f32c02175030bb3a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/SeaTunnelServer.java#L167 > > Perhaps we should modify it like this: > > 1. Remove the checkpoint storage creation every time the CheckpointManager is created, and reuse the created checkpoint storage. > https://github.com/apache/seatunnel/blob/3836c97a62079f1e8d7ca7a9f32c02175030bb3a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointManager.java#L102 > 2. When a job does not have checkpoints enabled, skip reading data from the checkpoint storage, which can reduce unnecessary overhead. > https://github.com/apache/seatunnel/blob/3836c97a62079f1e8d7ca7a9f32c02175030bb3a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointManager.java#L121 > > and > https://github.com/apache/seatunnel/blob/3836c97a62079f1e8d7ca7a9f32c02175030bb3a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointManager.java#L245 > > Regarding the logic of skipping checkpoint storage reading, for the sake of subsequent maintainability, we can inject a storage named empty into the checkpoint manager when the checkpoint is disabled, and all read and write operations will have no additional effect. OK, maybe I misunderstood. I think if it is not enabled, there is no need to trigger the relevant logic. Since this is the framework design, we can skip this issue. thx bro -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
