[
https://issues.apache.org/jira/browse/FLINK-8946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16501068#comment-16501068
]
Nikolas Davis commented on FLINK-8946:
--------------------------------------
I think we are seeing a similar issue. We are using Flink with Mesos. After our
Job Manager unexpectedly cycled, we are losing relevant Task Manager JVM
metrics. The TMs continue to report _other_ metrics, like operator related ones.
Here are the metric names I see disappearing (sorry, I'm unable to get a good
diff of fully qualified metric IDs):
{noformat}
1,8d0
< AvailableMemorySegments
< ClassesLoaded
< ClassesUnloaded
< Committed
< Count
< Load
< Max
< MemoryUsed
10,13d1
< Time
< TotalCapacity
< TotalMemorySegments
< Used{noformat}
At first glance these appear mostly to do with memory / JVM, which I suppose
are TM-specific.
More info can be found in the [mailing list
thread|https://www.mail-archive.com/user@flink.apache.org/msg19289.html].
> TaskManager stop sending metrics after JobManager failover
> ----------------------------------------------------------
>
> Key: FLINK-8946
> URL: https://issues.apache.org/jira/browse/FLINK-8946
> Project: Flink
> Issue Type: Bug
> Components: Metrics, TaskManager
> Affects Versions: 1.4.2
> Reporter: Truong Duc Kien
> Assignee: vinoyang
> Priority: Critical
> Fix For: 1.5.1
>
>
> Running in Yarn-standalone mode, when the Job Manager performs a failover,
> all TaskManager that are inherited from the previous JobManager will not send
> metrics to the new JobManager and other registered metric reporters.
>
> A cursory glance reveal that these line of code might be the cause
> [https://github.com/apache/flink/blob/a3478fdfa0f792104123fefbd9bdf01f5029de51/flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala#L1082-L1086]
> Perhap the TaskManager close its metrics group when disassociating
> JobManager, but not creating a new one on fail-over association ?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
