Thanks for Chesnay starting this discussion. In FLINK-17166 implementation[1], we are trying to use "tee" instead of introducing the stream redirection(redirect the out/err to files). However, a side effect is that the logging will be duplicated both in .log and .out files. Then it may consume more disk space. However it is not a very critical problem since we could use log4j/logback configuration to control the rolling files and max size.
Also, it only happens in docker/K8s deployment. For YARN/Mesos deployment, the behavior is just same as before. [1]. https://github.com/apache/flink/pull/11839 Best, Yang Chesnay Schepler <ches...@apache.org> 于2020年4月29日周三 上午12:30写道： > Currently, processes started in the foreground (like in the case of > Docker) output all logging/stdout directly to the console, without > creating any logging files. > > The downside of this approach, as outlined in FLIP-111, is that the > WebUI is not able to display the logs since it relies on these very > files to exist. > > In FLINK-17166 (part of FLIP-111) we are trying to change this such that > we always created .log/.out files. It seems like a reasonable change to > do, but it could have repercussions on existing deployments since we > will naturally use more disk space (logs gotta go somewhere). > > I'm curious what people think about this. > >
