@Patrick could you chime in? We should at least understand the original motivation before simply changing the way it works.
On Tue, May 5, 2020 at 2:49 PM Stephan Ewen <se...@apache.org> wrote: > I think Patrick originally introduced the foreground mode, and I believe > it had indeed something to do with container use and logging. > > IIRC the default assumption in docker and Kubernetes is that the logs come > on stdout (or stderr) so after "principle of least astonishment" the idea > was to give a similar experience with Flink. > > On Tue, May 5, 2020 at 10:49 AM David Anderson <da...@alpinegizmo.com> > wrote: > >> I like this idea because it should improve the experience (and reduce >> confusion) for folks having their first flink experience via one of the >> docker playgrounds. Right now it gives the impression that something is >> broken out-of-the-box. >> >> Regards, >> David >> >> On Mon, May 4, 2020 at 6:01 PM Till Rohrmann <trohrm...@apache.org> >> wrote: >> >> > Hi everyone, >> > >> > thanks for starting this discussion Chesnay. >> > >> > I think it would be nice if we also displayed the logs when starting the >> > process in the foreground. >> > >> > The repercussions could be mitigated if the default logger >> configurations >> > would contain file rolling with a max log file size. >> > >> > @Yang I think there are solutions how to redirect stdout and stderr into >> > separate files using tee without duplication [1]. >> > >> > [1] http://www.softpanorama.org/Tools/tee.shtml >> > >> > Cheers, >> > Till >> > >> > On Wed, Apr 29, 2020 at 4:28 AM Yang Wang <danrtsey...@gmail.com> >> wrote: >> > >> > > Thanks for Chesnay starting this discussion. >> > > >> > > In FLINK-17166 implementation[1], we are trying to use "tee" instead >> of >> > > introducing the stream redirection(redirect the out/err to files). >> > However, >> > > a side effect is that the logging will be duplicated both in .log and >> > .out >> > > files. >> > > Then it may consume more disk space. However it is not a very critical >> > > problem since we could use log4j/logback configuration to control the >> > > rolling >> > > files and max size. >> > > >> > > Also, it only happens in docker/K8s deployment. For YARN/Mesos >> > deployment, >> > > the behavior is just same as before. >> > > >> > > >> > > [1]. https://github.com/apache/flink/pull/11839 >> > > >> > > Best, >> > > Yang >> > > >> > > Chesnay Schepler <ches...@apache.org> 于2020年4月29日周三 上午12:30写道： >> > > >> > > > Currently, processes started in the foreground (like in the case of >> > > > Docker) output all logging/stdout directly to the console, without >> > > > creating any logging files. >> > > > >> > > > The downside of this approach, as outlined in FLIP-111, is that the >> > > > WebUI is not able to display the logs since it relies on these very >> > > > files to exist. >> > > > >> > > > In FLINK-17166 (part of FLIP-111) we are trying to change this such >> > that >> > > > we always created .log/.out files. It seems like a reasonable >> change to >> > > > do, but it could have repercussions on existing deployments since we >> > > > will naturally use more disk space (logs gotta go somewhere). >> > > > >> > > > I'm curious what people think about this. >> > > > >> > > > >> > > >> > >> >
