Flink Slot request bulk is not fulfillable!

2024-07-21 Thread Li Shao
Hi All, We are using flink batch mode to process s3 files. However, recently we are seeing the errors like: Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the

Re: Flink state

2024-07-21 Thread banu priya
Dear Community, Gentle reminder about my below email. Thanks Banu On Sat, 20 Jul, 2024, 4:37 pm banu priya, wrote: > Hi All, > > I have a flink job with RMQ Source, filters, tumbling window(uses > processing time fires every 2s), aggregator, RMQ Sink. > > I am trying to understand about states

Re: flight buffer local storage

2024-07-21 Thread Zhanghao Chen
By default, Flink uses aligned checkpoint where we wait for all in-flight data before the barriers to be fully processed and then make the checkpoints. There's no in need to store in-flight buffers in this case at the cost of additional barrier alignment, which may take a long time at the presen

Re: SavepointReader: Record size is too large for CollectSinkFunction

2024-07-21 Thread Zhanghao Chen
Hi, you could increase it as follows: Configuration config = new Configuration(); config.setString(collect-sink.batch-size.max, "10mb"); StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(config); From: Salva Alcántara Sent: Satur