I see I see. Thank you so much!
From: Xintong Song
Sent: Wednesday, April 29, 2020 11:22 PM
To: Jiahui Jiang
Cc: user@flink.apache.org
Subject: Re: Configuring taskmanager.memory.task.off-heap.size in Flink 1.10
That's pretty much it. I'm not very fam
them, we need to set up a non-zero task.off-heap?
>
> Thanks!
> --
> *From:* Xintong Song
> *Sent:* Wednesday, April 29, 2020 10:53 PM
> *To:* Jiahui Jiang
> *Cc:* Steven Wu ; user@flink.apache.org <
> user@flink.apache.org>
> *Subj
usage within them, we need to set up a non-zero task.off-heap?
Thanks!
From: Xintong Song
Sent: Wednesday, April 29, 2020 10:53 PM
To: Jiahui Jiang
Cc: Steven Wu ; user@flink.apache.org
Subject: Re: Configuring taskmanager.memory.task.off-heap.size in Flink 1.1
to do!
>
> I will test our some more pipeline with different resource requirements to
> understand the memory profiles better! Thank you guys again!
>
> --
> *From:* Steven Wu
> *Sent:* Wednesday, April 29, 2020 10:12 AM
> *To:* Xintong Song
tong Song
Cc: Jiahui Jiang ; user@flink.apache.org
Subject: Re: Configuring taskmanager.memory.task.off-heap.size in Flink 1.10
Jiahui,
Based on my reading on the doc, for containerized environment, it is probably
better to set `taskmanager.memory.process.size` to the container memory limit.
Jiahui,
Based on my reading on the doc, for containerized environment, it is
probably better to set `taskmanager.memory.process.size` to the container
memory limit.
https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#taskmanager-memory-process-size
Then I typically set `taskman
Hi Jiahui,
'taskmanager.memory.task.off-heap.size' accounts for the off-heap memory
reserved for your job / operators. There are other configuration options
accounting for the off-heap memory usages for other purposes, e.g.,
'taskmanager.memory.framework.off-heap'. The default 'task.off-heap.size'
Hello! We are migrating our pipeline from Flink 1.8 to 1.10 in Kubernetes.
In the first try, we simply copied the old 'taskmanager.heap.size' over to
'taskmanager.memory.flink.size'. This caused the cluster to OOM.
Eventually we had to allocate a small amount of memory to
'taskmanager.memory.tas
