Re:Re: Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-19 Thread Hirson Zhang
Hello Fu, I have tried "unalign checkpoint" before I ask, but it seems to effect only in "AT_LEAST_ONCE" mode. in my thought, "AT_LEAST_ONCE" mode makes slower checkpointing right? And I restart the job still, the first checkpoint is slower indeed. At 2025-05-16 13:52:44, "Di

Re: Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-15 Thread Dian Fu
It uses aligned checkpoint by default in Flink which needs to process all the data buffered in the pipeline(network and operators) during checkpointing. In your use case, as the process speed is very slow and so it may take too long to process the buffered data. You could try to enable unalign chec

Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread Sharath
Hello, Have you tried enabling the buffer debloating feature to improve checkpoint times? Refer taskmanager.network.memory.buffer-debloat.enabled in https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/ Regards, Sharath On Tue, May 13, 2025 at 1:59 AM 张河川 wrote: > Hi Fli