Hi guys, I'm running FLink on EMR with 2 m3.xlarge (each 16 GB RAM) and trying to process 3.8 GB CSV data from S3. I'm surprised the fact that Flink failed with OutOfMemory: Java Heap space
I tried to find the reason: 1) to identify TaskManager with a command ps aux | grep TaskManager 2) then build Heap histo: $ jmap -histo:live 19648 | head -n23 num #instances #bytes class name ---------------------------------------------- 1: 131018 3763501304 [B 2: 61022 7820352 <methodKlass> 3: 61022 7688456 <constMethodKlass> 4: 4971 5454408 <constantPoolKlass> 5: 4966 4582232 <instanceKlassKlass> 6: 4169 3003104 <constantPoolCacheKlass> 7: 15696 1447168 [C 8: 1291 638824 [Ljava.lang.Object; 9: 5318 506000 java.lang.Class Do you have any ideas what can be the reason and how it can be fixed? Is Flink uses out-of-heap memory? Thank you, Konstantin Kudryavtsev
