Re: Broadcast state and OutOfMemoryError: Direct buffer memory

2022-10-21 Thread yanfei lei
Hi Dan, Usually broadcast state needs more network buffers, the network buffer used to exchange data records between tasks would request a portion of direct memory[1], I think it is possible to get the “Direct buffer memory” OOM errors in this scenarios. Maybe you can try to increase taskmanager.m

Broadcast state and OutOfMemoryError: Direct buffer memory

2022-10-21 Thread Dan Hill
Hi. My team recently added broadcast state to our Flink jobs. We've started hitting this OOM "Direct buffer memory" error. Is this a common problem with broadcast state? Or is it likely a different problem? Thanks! - Dan