You don't have to enable the logging thread. You can also get the metrics of the job manager via the job manager web frontend. There, they also available in a JSON representation. So if you want, you can periodically (say every 5 seconds) do a HTTP request to get the metrics of all TMs.
On Mon, Jun 15, 2015 at 6:03 AM, Tamara Mendt <tammyme...@gmail.com> wrote: > Ok great, I will try this out and get back to you. Thanks =) > > On Mon, Jun 15, 2015 at 2:52 PM, Till Rohrmann <trohrm...@apache.org> > wrote: > >> Hi Tamara, >> >> you can instruct Flink to write the current memory statistics to the log >> by setting taskmanager.debug.memory.startLogThread: true in the Flink >> configuration. Furthermore, you can control the logging interval with >> taskmanager.debug.memory.logIntervalMs where the interval is specified >> in milli seconds. >> >> Cheers, >> Till >> >> On Mon, Jun 15, 2015 at 2:46 PM Fabian Hueske fhue...@gmail.com >> <http://mailto:fhue...@gmail.com> wrote: >> >> Hi Tamara, >>> >>> what kind of information do you need? Something like, size and usage of >>> in-memory sort buffers or hash tables? >>> Some information might written in DEBUG logs, but I'm not sure about >>> that. Besides logs, I doubt that Flink monitors memory usage. >>> >>> Cheers, Fabian >>> >>> 2015-06-15 14:34 GMT+02:00 Tamara Mendt <tammyme...@gmail.com>: >>> >>>> Hi, >>>> >>>> I am running some experiments on Flink and was wondering if there is >>>> some way to monitor the memory usage of a Flink Job (running locally and on >>>> a cluster). I need to run multiple jobs and compare their memory usage. >>>> >>>> Cheers, >>>> >>>> Tamara >>>> >>> >>> >> > > > > -- > Tamara Mendt >
