Hi Akshay,
Sorrry I have not thought of a proper way to handle single large record in
distributed task managers in flink. But I can give some hints for adjusting the
related memories for work around OOM issue.
Large fraction of memories in task manager are managed by flink for efficiency,
and t
Hi Akshay,
You encountered an existing issue for serializing large records to cause OOM.
Every subpartition would create a separate serializer before, and each
serializer would maintain an internal bytes array for storing intermediate
serialization results. The key point is that these overhead
