[
https://issues.apache.org/jira/browse/FLINK-17493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106422#comment-17106422
]
Oisín Mac Fhearaí edited comment on FLINK-17493 at 5/13/20, 4:05 PM:
---------------------------------------------------------------------
Some more accurate memory stats (grepping "Off-heap pool stats" in the log,
while the job crashes and restarts in the background):
{code:java}
2020-05-13 15:06:36,380 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 52/53/240 MB (used/committed/max)], [Metaspace:
440/448/2048 MB (used/committed/max)], [Compressed Class Space: 58/59/1024 MB
(used/committed/max)]
2020-05-13 15:11:36,380 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 66/66/240 MB (used/committed/max)], [Metaspace:
711/723/2048 MB (used/committed/max)], [Compressed Class Space: 94/96/1024 MB
(used/committed/max)]
2020-05-13 15:16:36,380 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 77/78/240 MB (used/committed/max)], [Metaspace:
965/981/2048 MB (used/committed/max)], [Compressed Class Space: 128/131/1024 MB
(used/committed/max)]
2020-05-13 15:21:36,381 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 87/88/240 MB (used/committed/max)], [Metaspace:
1202/1222/2048 MB (used/committed/max)], [Compressed Class Space: 160/163/1024
MB (used/committed/max)]
2020-05-13 15:26:36,664 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 98/98/240 MB (used/committed/max)], [Metaspace:
1439/1462/2048 MB (used/committed/max)], [Compressed Class Space: 192/196/1024
MB (used/committed/max)]
2020-05-13 15:31:36,665 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 106/107/240 MB (used/committed/max)], [Metaspace:
1642/1668/2048 MB (used/committed/max)], [Compressed Class Space: 219/224/1024
MB (used/committed/max)]
2020-05-13 15:36:36,665 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 115/116/240 MB (used/committed/max)], [Metaspace:
1845/1875/2048 MB (used/committed/max)], [Compressed Class Space: 246/251/1024
MB (used/committed/max)]
2020-05-13 15:41:36,666 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 122/123/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
2020-05-13 15:46:43,113 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 122/123/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
2020-05-13 15:51:46,591 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 123/124/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
2020-05-13 15:56:46,592 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 123/124/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
{code}
was (Author: destynova):
Some more accurate memory stats (grepping "Off-heap pool stats" in the log):
{code}
2020-05-13 15:06:36,380 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 52/53/240 MB (used/committed/max)], [Metaspace:
440/448/2048 MB (used/committed/max)], [Compressed Class Space: 58/59/1024 MB
(used/committed/max)]
2020-05-13 15:11:36,380 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 66/66/240 MB (used/committed/max)], [Metaspace:
711/723/2048 MB (used/committed/max)], [Compressed Class Space: 94/96/1024 MB
(used/committed/max)]
2020-05-13 15:16:36,380 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 77/78/240 MB (used/committed/max)], [Metaspace:
965/981/2048 MB (used/committed/max)], [Compressed Class Space: 128/131/1024 MB
(used/committed/max)]
2020-05-13 15:21:36,381 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 87/88/240 MB (used/committed/max)], [Metaspace:
1202/1222/2048 MB (used/committed/max)], [Compressed Class Space: 160/163/1024
MB (used/committed/max)]
2020-05-13 15:26:36,664 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 98/98/240 MB (used/committed/max)], [Metaspace:
1439/1462/2048 MB (used/committed/max)], [Compressed Class Space: 192/196/1024
MB (used/committed/max)]
2020-05-13 15:31:36,665 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 106/107/240 MB (used/committed/max)], [Metaspace:
1642/1668/2048 MB (used/committed/max)], [Compressed Class Space: 219/224/1024
MB (used/committed/max)]
2020-05-13 15:36:36,665 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 115/116/240 MB (used/committed/max)], [Metaspace:
1845/1875/2048 MB (used/committed/max)], [Compressed Class Space: 246/251/1024
MB (used/committed/max)]
2020-05-13 15:41:36,666 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 122/123/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
2020-05-13 15:46:43,113 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 122/123/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
2020-05-13 15:51:46,591 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 123/124/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
2020-05-13 15:56:46,592 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Off-heap pool
stats: [Code Cache: 123/124/240 MB (used/committed/max)], [Metaspace:
2002/2048/2048 MB (used/committed/max)], [Compressed Class Space: 267/275/1024
MB (used/committed/max)]
{code}
> Possible direct memory leak in cassandra sink
> ---------------------------------------------
>
> Key: FLINK-17493
> URL: https://issues.apache.org/jira/browse/FLINK-17493
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Cassandra
> Affects Versions: 1.9.3, 1.10.0
> Reporter: nobleyd
> Priority: Major
>
> # Cassandra Sink use direct memorys.
> # Start a standalone cluster(1 machines) for test.
> # After the cluster started, check the flink web-ui, and record the task
> manager's memory info. I mean the direct memory part info.
> # Start a job which read from kafka and write to cassandra using the
> cassandra sink, and you can see that the direct memory count in 'Outside JVM'
> part go up.
> # Stop the job, and the direct memory count is not decreased(using 'jmap
> -histo:live pid' to make the task manager gc).
> # Repeat serveral times, the direct memory count will be more and more.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
