Sure, /opt/flink/conf is mounted as a volume from the configmap. Best Yun Tang
From: Li Peng <li.p...@doordash.com> Date: Wednesday, December 11, 2019 at 9:37 AM To: Yang Wang <danrtsey...@gmail.com> Cc: vino yang <yanghua1...@gmail.com>, user <user@flink.apache.org> Subject: Re: Flink on Kubernetes seems to ignore log4j.properties 1. Hey Yun, I'm calling /opt/flink/bin/standalone-job.sh and /opt/flink/bin/taskmanager.sh on my job and task managers respectively. It's based on the setup described here: http://shzhangji.com/blog/2019/08/24/deploy-flink-job-cluster-on-kubernetes/ . I haven't tried the configmap approach yet, does it also replace the conf files in /opt/flink/conf? 2. Hey Vino, here's a sample of the kubernetes: https://pastebin.com/fqJrgjZu I didn't change any patterns from the default, so the string patterns should look the same, but as you can see it's full of info checkpoint logs that I originally was trying to suppress. Based on my log4j.properties, the level should be set to WARN. I couldn't actually find any .out files on the pod, this is from the kubectl logs command. I also didn't see any files in /opt/flink/log, which I thought my log4j was specified to do, hence me thinking that the properties weren't actually being consumed. I also have the same properties in my src/main/resources folder. 3. Hey Yang, yes this is a standalone session cluster. I did specify in the docker file to copy the log4j.properties to the /opt/flink/conf folder on the image, and I confirmed that the properties are correct when I bash'd into the pod and viewed them manually. Incidentally, I also tried passing the -Dlog4j.configuration argument to the programs, and it doesn't work either. And based on what I'm reading on jira, that option is not really supported anymore? Thanks for your responses, folks! Li On Mon, Dec 9, 2019 at 7:10 PM Yang Wang <danrtsey...@gmail.com<mailto:danrtsey...@gmail.com>> wrote: Hi Li Peng, You are running standalone session cluster or per-job cluster on kubernetes. Right? If so, i think you need to check your log4j.properties in the image, not local. The log is stored to /opt/flink/log/jobmanager.log by default. If you are running active Kubernetes integration for a fresh taste. The following cli option could be used to remove the redirect. -Dkubernetes.container-start-command-template="%java% %classpath% %jvmmem% %jvmopts% %logging% %class% %args%" Best, Yang vino yang <yanghua1...@gmail.com<mailto:yanghua1...@gmail.com>> 于2019年12月10日周二 上午10:55写道： Hi Li, A potential reason could be conflicting logging frameworks. Can you share the log in your .out file and let us know if the print format of the log is the same as the configuration file you gave. Best, Vino Li Peng <li.p...@doordash.com<mailto:li.p...@doordash.com>> 于2019年12月10日周二 上午10:09写道： Hey folks, I noticed that my kubernetes flink logs (reached via kubectl logs <pod-name>) completely ignore any of the configurations I put into /flink/conf/. I set the logger level to WARN, yet I still see INFO level logging from flink loggers like org.apache.flink.runtime.checkpoint.CheckpointCoordinator. I even made copied the same properties to /flink/conf/log4j-console.properties and log4j-cli.properties. From what I can tell, kubernetes just listens to stdout and stderr, so shouldn't the log4j.properties control output to them? Anyone seen this issue before? Here is my log4j.properties: # This affects logging for both user code and Flink log4j.rootLogger=WARN, file, console, stdout # Uncomment this if you want to _only_ change Flink's logging log4j.logger.org.apache.flink=WARN # The following lines keep the log level of common libraries/connectors on # log level INFO. The root logger does not override this. You have to manually # change the log levels here. log4j.logger.akka=INFO log4j.logger.org.apache.kafka=INFO log4j.logger.org.apache.hadoop=INFO log4j.logger.org.apache.zookeeper=INFO # Log all infos in the given file log4j.appender.file=org.apache.log4j.FileAppender log4j.appender.file.file=${log.file} log4j.appender.file.append=false log4j.appender.file.layout=org.apache.log4j.PatternLayout log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n # Log all infos to the console log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n # Suppress the irrelevant (wrong) warnings from the Netty channel handler log4j.logger.org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline=ERROR, file, console log4j.logger.org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction=WARN log4j.logger.org.apache.flink.runtime.checkpoint=WARN Thanks, Li
