Hi,
don't know if this is the problem you're facing, but some time ago we
encountered two issues connected to REST API and increased disk usage
after each submission:
https://issues.apache.org/jira/browse/FLINK-21164
https://issues.apache.org/jira/browse/FLINK-9844
- they're closed ATM, but only 1.12.2 contains the fixes.
maciek
On 08.04.2021 19:52, Great Info wrote:
I have deployed my own flink setup in AWS ECS. One Service for
JobManager and one Service for task Managers. I am running one ECS
task for a job manager and 3 ecs tasks for TASK managers.
I have a kind of batch job which I upload using flink rest every-day
with changing new arguments, when I submit each time disk memory gets
increased by ~ 600MB, I have given a checkpoint as S3 . Also I have
set *historyserver.archive.clean-expired-jobs* true .
Since I am running on ECS, I am not able to find why the memory is
getting increased on every jar upload and execution .
What are the flink config params I should look at to make sure the
memory is not shooting up?
