Flink support for data compression for pipelined shuffle

2025-01-28 Thread Kanchi Masalia via user
Hi, I'm exploring data compression for a Flink 1.18 job in streaming mode (pipeline shuffle) and noticed that it's currently supported only in batch mode (blocking and hybrid shuffle). I found that support for this in streaming mode was added FLINK-14845

Re: table.exec.source.idle-timeout support

2025-01-28 Thread Nikola Milutinovic
Hi Zhao. Yes, I am deploying our solution as a Session Cluster on Kubernetes, via Flink Kubernetes Operator. So, the JM is the session cluster leader. So, what is the correct approach? Should I explicitly set these checkpointing properties in my jobs? Funny thing is, it used to work on the sam