Ajinkya Pathrudkar <ajinkya.pathrudka...@gmail.com> 10:53 AM (1 hour ago) to user-info I hope this email finds you well. I am writing to inform you of a recent update we made to our Flink version, upgrading from 1.14 to 1.15, along with a shift from Java 8 to Java 11. Since the update, we have encountered out-of-memory (direct memory) exceptions.
The exception message suggests that increasing the task-off-heap memory could resolve the issue, but we are uncertain if there have been any changes to the memory model that may be contributing to this problem. The error message we received is as follows: java.lang.OutOfMemoryError: Can't allocate enough direct buffer for batch shuffle read buffer pool (bytes allocated: 26017792, bytes still needed: 41091072). To avoid the exception, you need to do one of the following adjustments: 1) If you have ever decreased taskmanager.memory.framework.off-heap.size, you need to undo the decrement; 2) If you ever increased taskmanager.memory.framework.off-heap.batch-shuffle.size, you should also increase taskmanager.memory.framework.off-heap.size; 3) If neither the above cases, it usually means some other parts of your application have consumed too many direct memory and the value of taskmanager.memory.task.off-heap.size should be increased. We would appreciate any insights you may have regarding this issue, and any suggestions on how to proceed would be greatly appreciated. Please let us know if there are any additional details we can provide to assist you in diagnosing the issue. Thank you for your time and consideration. Thanks, Ajinkya
