Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread
Hi Flink community, I’m encountering an issue with PyFlink where a FlatMap operator invokes an external service (using a PyTorch model to generate embedding vectors). The operator processes data very slowly, leading to an extremely long initial checkpoint start delay, which eventually causes ch

Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread
Hi Flink community, I’m encountering an issue with PyFlink where a FlatMap operator invokes an external service (using a PyTorch model to generate embedding vectors). The operator processes data very slowly, leading to an extremely long initial checkpoint start delay, which eventually causes ch

Re:Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread
checkpoint >times? Refer taskmanager.network.memory.buffer-debloat.enabled in >https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/ > >Regards, >Sharath > >On Tue, May 13, 2025 at 1:59 AM 张河川 wrote: > >> Hi Flink community, >> >> I’m encountering an issue wi