Hi Jiahui, 'taskmanager.memory.task.off-heap.size' accounts for the off-heap memory reserved for your job / operators. There are other configuration options accounting for the off-heap memory usages for other purposes, e.g., 'taskmanager.memory.framework.off-heap'. The default 'task.off-heap.size' being 0 only represents that in most cases user codes / operators do not use off-heap memory. User would need to explicitly increase this configuration if UDFs or libraries of the job uses off-heap memory.
Thank you~ Xintong Song On Wed, Apr 29, 2020 at 11:07 AM Jiahui Jiang <qzhzm173...@hotmail.com> wrote: > Hello! We are migrating our pipeline from Flink 1.8 to 1.10 in Kubernetes. > > In the first try, we simply copied the old 'taskmanager.heap.size' over to > 'taskmanager.memory.flink.size'. This caused the cluster to OOM. > Eventually we had to allocate a small amount of memory to > 'taskmanager.memory.task.off-heap.size' for it to stop failing. But we > don't quite understand why this needs to be overriden. > > I saw the default for 'taskmanager.memory.task.off-heap.size' is 0, does > that mean in most cases task managers won't need off-heap memory? What are > some examples that off-heap memory need to be non-zero? > > Thank you! >
