Hi everyone, thanks for starting this discussion Chesnay.
I think it would be nice if we also displayed the logs when starting the process in the foreground. The repercussions could be mitigated if the default logger configurations would contain file rolling with a max log file size. @Yang I think there are solutions how to redirect stdout and stderr into separate files using tee without duplication [1]. [1] http://www.softpanorama.org/Tools/tee.shtml Cheers, Till On Wed, Apr 29, 2020 at 4:28 AM Yang Wang <danrtsey...@gmail.com> wrote: > Thanks for Chesnay starting this discussion. > > In FLINK-17166 implementation[1], we are trying to use "tee" instead of > introducing the stream redirection(redirect the out/err to files). However, > a side effect is that the logging will be duplicated both in .log and .out > files. > Then it may consume more disk space. However it is not a very critical > problem since we could use log4j/logback configuration to control the > rolling > files and max size. > > Also, it only happens in docker/K8s deployment. For YARN/Mesos deployment, > the behavior is just same as before. > > > [1]. https://github.com/apache/flink/pull/11839 > > Best, > Yang > > Chesnay Schepler <ches...@apache.org> 于2020年4月29日周三 上午12:30写道： > > > Currently, processes started in the foreground (like in the case of > > Docker) output all logging/stdout directly to the console, without > > creating any logging files. > > > > The downside of this approach, as outlined in FLIP-111, is that the > > WebUI is not able to display the logs since it relies on these very > > files to exist. > > > > In FLINK-17166 (part of FLIP-111) we are trying to change this such that > > we always created .log/.out files. It seems like a reasonable change to > > do, but it could have repercussions on existing deployments since we > > will naturally use more disk space (logs gotta go somewhere). > > > > I'm curious what people think about this. > > > > >
