Hi Yun, thank you for the advice, but how would you suggest doing it to get the metrics also for each TaskManager? I do not urgently need to use REST because I’m running my code within Flink. Maybe there is another way to access it?
Thanks a lot. Benjamin Am 2. Apr. 2019, 18:26 +0200 schrieb Yun Tang <myas...@live.com>: > Hi Benjamin > > Try this > http://localhost:8081/jobs/{job-id}/vertices/{vertices-id}/subtasks/{subtask-index}/metrics?get=numBytesInLocalPerSecond > > You could GET http://localhost:8081/jobs/ to know running jobs, and GET > http://localhost:8081/jobs/{job-id}/vertices/ to know all vertices similarly. > > However, AFAIK, if you use REST API to query I'm afraid you cannot directly > know the received records per task manager, and you have to gather these > metrics per task. > > Best > Yun Tang > From: Benjamin Burkhardt <pri...@benjaminburkhardt.de> > Sent: Tuesday, April 2, 2019 21:56 > To: user@flink.apache.org; Yun Tang > Subject: Re: Metrics for received records per TaskManager > > Hi Yun, > > thanks for the hint. I tried to access the metric through the REST API > calling > http://localhost:8081/taskmanagers/2264f296385854f2d1fb4d121495822a/metrics?get= > numBytesInRemotePerSecond. > > Unfortunately the metric is not available... > > Only these are avaiblable: > [{"id":"Status.Network.AvailableMemorySegments"},{"id":"Status.JVM.Memory.NonHeap.Committed"},{"id":"Status.JVM.Memory.Mapped.TotalCapacity"},{"id":"Status.JVM.Memory.NonHeap.Used"},{"id":"Status.JVM.GarbageCollector.G1_Old_Generation.Count"},{"id":"Status.Network.TotalMemorySegments"},{"id":"Status.JVM.Memory.Direct.MemoryUsed"},{"id":"Status.JVM.Memory.Mapped.MemoryUsed"},{"id":"Status.JVM.CPU.Time"},{"id":"Status.JVM.GarbageCollector.G1_Young_Generation.Count"},{"id":"Status.JVM.Threads.Count"},{"id":"Status.JVM.GarbageCollector.G1_Old_Generation.Time"},{"id":"Status.JVM.Memory.Direct.TotalCapacity"},{"id":"Status.JVM.Memory.Heap.Committed"},{"id":"Status.JVM.ClassLoader.ClassesLoaded"},{"id":"Status.JVM.Memory.Mapped.Count"},{"id":"Status.JVM.Memory.Direct.Count"},{"id":"Status.JVM.CPU.Load"},{"id":"Status.JVM.Memory.Heap.Used"},{"id":"Status.JVM.Memory.Heap.Max"},{"id":"Status.JVM.ClassLoader.ClassesUnloaded"},{"id":"Status.JVM.GarbageCollector.G1_Young_Generation.Time"},{„id“:“Status.JVM.Memory.NonHeap.Max“}] > > > How do I enable it, maybe in the flink-conf? > > Thanks. > > Benjamin > Am 2. Apr. 2019, 10:37 +0200 schrieb Yun Tang <myas...@live.com>: > > Hi Benjamin > > > > I think 'numBytesInLocalPerSecond' and 'numBytesInRemotePerSecond' which > > indicate 'The number of bytes this task reads from a local source per > > second' and 'The number of bytes this task reads from a remote source per > > second' respectively could help you. If you want to track the information > > by each TaskManager, please group the metrics by tag 'tm_id'. > > > > You could refer to > > https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#io > > for more information. > > > > Best > > Yun Tang > > > > From: Benjamin Burkhardt <pri...@benjaminburkhardt.de> > > Sent: Tuesday, April 2, 2019 15:00 > > To: user@flink.apache.org > > Subject: Metrics for received records per TaskManager > > > > Hi all, > > > > I’m looking for a metric which allows me keeping track of the records or > > bytes each TaskManager has received or processed for the current task. > > > > Can anyone help me getting this? > > > > Thanks. > > > > Benjamin
