Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-22 Thread Andrey Zagrebin
Thanks for summarising the discussion points, Till. # Configuration ## Env variables Agree, this looks like an independent effort. ## dynamic program arguments Indeed, jobmanager.sh needs small extension. It can be addressed independently but I think it has chance to be addressed in this release

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-18 Thread Till Rohrmann
Thanks for creating this FLIP Andrey. I like the general idea pretty much. I tried to group some of the above-mentioned points in order to give my 50 cent. # Configuration How to configure the Flink process seems to be the biggest question. Due to historical reasons we have a plethora of differe

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-17 Thread Yangze Guo
I second Thomas that we can support both Java 8 and 11. Best, Yangze Guo On Wed, Mar 18, 2020 at 12:12 PM Thomas Weise wrote: > > --> > > On Mon, Mar 16, 2020 at 1:58 AM Andrey Zagrebin wrote: >> >> Thanks for the further feedback Thomas and Yangze. >> >> > A generic, dynamic configuration mech

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-17 Thread Thomas Weise
--> On Mon, Mar 16, 2020 at 1:58 AM Andrey Zagrebin wrote: > Thanks for the further feedback Thomas and Yangze. > > > A generic, dynamic configuration mechanism based on environment variables > is essential and it is already supported via envsubst and an environment > variable that can supply a

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-17 Thread Yang Wang
Hi Andrey, Thanks for your explanation. > About the logging What i mean is we could not forward the stdout/stderr to local files and docker stdout at the same time by using log4j. For the jobmanager.log/taskmanager.log, it works quite well since we only need to add a console appender in the log4j

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-16 Thread Andrey Zagrebin
Thanks for the further feedback Thomas and Yangze. > A generic, dynamic configuration mechanism based on environment variables is essential and it is already supported via envsubst and an environment variable that can supply a configuration fragment True, we already have this. As I understand thi

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-10 Thread Yangze Guo
Thanks for the reply, Andrey. Regarding building from local dist: - Yes, I bring this up mostly for development purpose. Since k8s is popular, I believe more and more developers would like to test their work on k8s cluster. I'm not sure should all developers write a custom docker file themselves i

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-10 Thread Thomas Weise
Thanks for working on improvements to the Flink Docker container images. This will be important as more and more users are looking to adopt Kubernetes and other deployment tooling that relies on Docker images. A generic, dynamic configuration mechanism based on environment variables is essential a

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-10 Thread Andrey Zagrebin
Hi All, Thanks a lot for the feedback! *@Yangze Guo* - Regarding the flink_docker_utils#install_flink function, I think it > should also support build from local dist and build from a > user-defined archive. I suppose you bring this up mostly for development purpose or powerful users. Most of n

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-08 Thread Konstantin Knauf
Hi Andrey, thanks a lot for this proposal. The variety of Docker files in the project has been causing quite some confusion. For the entrypoint, have you considered to also allow setting configuration via environment variables as in "docker run -e FLINK_REST_BIN_PORT=8081 ..."? This is quite comm

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-05 Thread Yang Wang
Hi Andrey, Thanks for driving this significant FLIP. From the user ML, we could also know there are many users running Flink in container environment. Then the docker image will be the very basic requirement. Just as you say, we should provide a unified place for all various usage(e.g. session,

[DISCUSS] FLIP-111: Docker image unification

2020-03-04 Thread Andrey Zagrebin
Hi All, If you have ever touched the docker topic in Flink, you probably noticed that we have multiple places in docs and repos which address its various concerns. We have prepared a FLIP [1] to simplify the perception of docker topic in Flink by users. It mostly advocates for an approach of exte