Hi Stephan, Till
I am looking for the memory usage info in flink_<user>_jobmanager... log.
As there are no other log files in /log dir I assumed this is the right
place to look. I will try to setup a new clean instance and check whether
it will work.
Thank you very much for your assistance!
Best regards,
Filip Łęczycki
Pozdrawiam,
Filip Łęczycki
2015-12-09 11:13 GMT+01:00 Stephan Ewen <se...@apache.org>:
> Hi Filip!
>
> Someone else just used the memory logging with the exact described
> settings - it worked.
>
> There is probably some mixup, you may be looking into the wrong log file,
> or may setting the a value in a different config...
>
> Stephan
>
>
> On Wed, Dec 9, 2015 at 10:58 AM, Till Rohrmann <trohrm...@apache.org>
> wrote:
>
>> I assume you're looking in the taskmanager log file for the memory usage
>> logging statements, right?
>>
>> Cheers,
>> Till
>>
>> On Wed, Dec 9, 2015 at 12:15 AM, Filip Łęczycki <filipleczy...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Thank you for your reply!
>>>
>>> I have made sure I restarted the TaskManager after changing config, but
>>> it didn't resolve the issue.The config is loaded as I can see the following
>>> line in the log:
>>> 09:12:2015 00:00:21,894 DEBUG
>>> org.apache.flink.configuration.GlobalConfiguration - Loading
>>> configuration property: taskmanager.debug.memory.startLogThread, true
>>>
>>> I am running a job on local standalone flink instance and my
>>> log4j.properties look like this:
>>> log4j.rootLogger=DEBUG, file
>>>
>>> # Log all infos in the given file
>>> log4j.appender.file=org.apache.log4j.FileAppender
>>> log4j.appender.file.file=${log.file}
>>> log4j.appender.file.append=false
>>> log4j.appender.file.layout=org.apache.log4j.PatternLayout
>>> log4j.appender.file.layout.ConversionPattern=%d{dd:MM:YYY HH:mm:ss,SSS}
>>> %-5p %-60c %x - %m%n
>>>
>>> # suppress the irrelevant (wrong) warnings from the netty channel handler
>>> log4j.logger.org.jboss.netty.channel.DefaultChannelPipeline=ERROR, file
>>>
>>> Maybe I can set some property in the job's code so that it would force
>>> such verbose logging? I need those logs to run some flink performance
>>> tests but maybe I can somehow extract the benchmark results without them,
>>> do
>>> you know any other way to monitor Flink Job's memory usage and GC time,
>>> other than looking at web interface?
>>>
>>> Best regards,
>>> Filip Łęczycki
>>>
>>> Pozdrawiam,
>>> Filip Łęczycki
>>>
>>> 2015-12-08 20:48 GMT+01:00 Stephan Ewen <se...@apache.org>:
>>>
>>>> Hi!
>>>>
>>>> That is exactly the right way to do it. Logging has to be at least INFO
>>>> and the parameter "taskmanager.debug.memory.startLogThread" set to true.
>>>> The log output should be under
>>>> "org.apache.flink.runtime.taskmanager.TaskManager".
>>>>
>>>> Do you see other outputs for that class in the log?
>>>>
>>>> Make sure you restarted the TaskManager processes after you changed the
>>>> config file.
>>>>
>>>> Greetings,
>>>> Stephan
>>>>
>>>>
>>>>
>>>> On Tue, Dec 8, 2015 at 6:56 PM, Filip Łęczycki <filipleczy...@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am trying to enable logging of memory usage on flink 0.10.0 by
>>>>> adding:
>>>>>
>>>>> taskmanager.debug.memory.startLogThread: true
>>>>>
>>>>> to config.yaml and setting log4j level to DEBUG however in the logs after
>>>>> running the job I cannot see any info regarding memory usage.My job
>>>>> lasted 30s so it should catch few intervals. Should I change something
>>>>> else in the configuration?
>>>>>
>>>>>
>>>>> Best regards/Pozdrawiam,
>>>>>
>>>>> Filip Łęczycki
>>>>>
>>>>
>>>>
>>>
>>
>
