Hi Stefanos, you can also write yourself a little script/tool which is periodically requesting the following JSON from the JobManager:
http://localhost:8081/setupInfo?get=taskmanagers&_=1438972693441 It returns a JSON string like this: {"taskmanagers":[{"path":"akka:\/\/flink\/user\/taskmanager","dataPort":51695,"timeSinceLastHeartbeat":3,"slotsNumber":1,"freeSlots":1,"cpuCores":8,"physicalMemory":15941,"freeMemory":736,"managedMemory":461,"instanceID":"187cdbc1de0acdb02b5bdc796cefceaf","metrics":{"version":"3.0.0","gauges":{"cpuLoad":{"value":0.09425878320479862},"gc.PS-MarkSweep.count":{"value":6},"gc.PS-MarkSweep.time":{"value":764},"gc.PS-Scavenge.count":{"value":3},"gc.PS-Scavenge.time":{"value":121},"load":{"value":3.04},"memory.heap.committed":{"value":771751936},"memory.heap.init":{"value":805306368},"memory.heap.max":{"value":771751936},"memory.heap.usage":{"value":0.741875586302384},"memory.heap.used":{"value":572543920},"memory.non-heap.committed":{"value":50790400},"memory.non-heap.init":{"value":24576000},"memory.non-heap.max":{"value":318767104},"memory.non-heap.usage":{"value":0.09098903756392629},"memory.non-heap.used":{"value":29004312},"memory.pools.Code-Cache.usage":{"value":0.020928700764973957},"memory.pools.PS-Eden-Space.usage":{"value":0.1786344051361084},"memory.pools.PS-Old-Gen.usage":{"value":0.9994582533836365},"memory.pools.PS-Perm-Gen.usage":{"value":0.10412535071372986},"memory.pools.PS-Survivor-Space.usage":{"value":0},"memory.total.committed":{"value":822542336},"memory.total.init":{"value":829882368},"memory.total.max":{"value":1090519040},"memory.total.used":{"value":601962568}},"counters":{},"histograms":{},"meters":{},"timers":{}}}]} On Fri, Aug 7, 2015 at 5:43 PM, Andra Lungu <lungu.an...@gmail.com> wrote: > Hi, > > You can use something like iostat to extract the CPU usage. > For instance, I call this script on the JM node: > #!/usr/bin/env bash > > lines=`cat /home/andra.lungu/hostnames.txt | paste -d, -s` > echo "${lines}" > IFS="," read -ra hostnames <<<"${lines}" > for line in "${hostnames[@]}"; do > ssh ${line} `iostat -xtc 5 3 > /home/andra.lungu/cpu_stats/${line}` & > done > > And then I parse the individual output in the cpu_stats folder with python > to form charts. > To my knowledge. Flink does not do something like that for you, at least > not yet :) > > Bottom line is that you can use linux commands: iostat, dstat, htop, etc. > Hope this helps! > Andra > > On Fri, Aug 7, 2015 at 5:07 PM, Stefanos Antaris < > antaris.stefa...@gmail.com> wrote: > >> Hi to all. >> >> I am working an a research project using flink and i would like to >> extract the CPU + RAM resources consumed on each worker in order to include >> the stats in my paper. Can anyone advice me on how could i extract them? >> >> Thanks in advance. >> Stefanos > > >
