Yun Gao created FLINK-13752:
-------------------------------
Summary: TaskDeploymentDescriptor cannot be recycled by GC due to
referenced by an anonymous function
Key: FLINK-13752
URL: https://issues.apache.org/jira/browse/FLINK-13752
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.9.0
Reporter: Yun Gao
When comparing the 1.8 and 1.9.0-rc2 on a test streaming job, we found that the
performance on 1.9.0-rc2 is much lower than that of 1.8. By comparing the two
versions, we found that the count of Full GC on 1.9.0-rc2 is much more than
that on 1.8.
A further analysis found that the difference is due to in
TaskExecutor#setupResultPartitionBookkeeping, the anonymous function in
taskTermimationWithResourceCleanFuture has referenced the
TaskDeploymentDescriptor, since this function will be kept till the task is
terminated, TaskDeploymentDescriptor will also be kept referenced in the
closure and cannot be recycled by GC. In this job, TaskDeploymentDescriptor of
some tasks are as large as 10M, and the total heap is about 113M, thus the kept
TaskDeploymentDescriptors will cause relatively large impact on GC and
performance.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
