Problem to show logs in task managers

Hi flink community, I am trying to show log messages using log4j. It works fine overall except for the messages I want to show in an inner class that implements org.apache.flink.api.common.aggregators.ConvergenceCriterion. I am very new to this, but it seems that I’m having problems to show the

Re: Problem to show logs in task managers

ohrmann mailto:trohrm...@apache.org>> wrote: Hi Ana, you can simply modify the `log4j.properties` file in the `conf` directory. It should be automatically included in the Yarn application. Concerning your logging problem, it might be that you have set the logging level too high. Co

Re: Problem to show logs in task managers

simply retrieve the log files via yarn logs. Cheers, Till On Fri, Dec 18, 2015 at 3:49 PM, Ana M. Martinez mailto:a...@cs.aau.dk>> wrote: Hi Till, Many thanks for your quick response. I have modified the WordCountExample to re-reproduce my problem in a simple example. I run the code be

Re: Problem to show logs in task managers

n, Jan 4, 2016 at 12:36 PM, Ana M. Martinez mailto:a...@cs.aau.dk>> wrote: Hi Till, Sorry for the delay (Xmas break). I have activated log aggregation on flink-conf.yaml with yarn.log-aggregation-enable: true (as I can’t find a yarn-site.xml). But the command yarn logs -a

Re: Problem to show logs in task managers

er - YARN Client is shutting down You can see the log messages from the WordCountExample and TestClass classes. But I have problems to show the logger message (INFO) in the LineSplitter class. Presumably, because it is executed in the CORE nodes and node in the MASTER node

Re: Problem to show logs in task managers

serlogs you should find somewhere the logs. Cheers, Till ​ On Fri, Jan 8, 2016 at 3:50 PM, Ana M. Martinez mailto:a...@cs.aau.dk>> wrote: Thanks for the tip Robert! It was a good idea to rule out other possible causes, but I am afraid that is not the problem. If we stick to the Word

Re: Problem to show logs in task managers

On Mon, Jan 11, 2016 at 9:37 AM, Ana M. Martinez mailto:a...@cs.aau.dk>> wrote: Hi Till, Thanks for that! I can see the "Logger in LineSplitter.flatMap” output if I retrieve the task manager logs manually (under /var/log/hadoop-yarn/containers/application_X/…). However that solu

Re: Problem to show logs in task managers

at via HADOOP_HOME/sbin/stop-yarn.sh; HADOOP_HOME/sbin/start-yarn.sh. The commands yarn-session.sh ... and bin/flink run -m yarn cluster start a new yarn application within the yarn cluster. Cheers, Till ​ On Mon, Jan 11, 2016 at 1:39 PM, Ana M. Martinez mailto:a...@cs.aau.dk>> wrote: Hi Till,

Could not upload the jar files to the job manager IOException

Hi all, I am running some experiments with flink in an Amazon cluster and every now and then (it seems to appear at random) I get the following IOException: > org.apache.flink.client.program.ProgramInvocationException: The program > execution failed: Could not upload the jar files to the job man

Re: Could not upload the jar files to the job manager IOException

you're trying to submit the job to? Maybe there you can find helpful information why it failed. On Wed, Jan 20, 2016 at 3:23 PM, Ana M. Martinez mailto:a...@cs.aau.dk>> wrote: Hi all, I am running some experiments with flink in an Amazon cluster and every now and then (it seems to appear

Use jvm to run flink on single-node machine with many cores

Hi all, I am trying to run a program using the flink java library with ExecutionEnvironment.getExecutionEnvironment() from the command line using java -jar. If I run the code in my machine (with four cores) or in a multi-node cluster (using yarn) the program runs normally, but if I want to run

Re: Use jvm to run flink on single-node machine with many cores

o the configuration I would personally recommend using the local cluster option in the flink distribution. [1] If you want to avoid having a flink distro on the machine, then Robert's solution is the way to go. [1] https://ci.apache.org/projects/flink/flink-docs-master/setup/local_setup.html On Sun