There was also a talk about containerization at Flink Forward that touches on some of your questions. https://www.youtube.com/watch?v=w721NI-mtAA&t=2s&list=PLDX4T_cnKjD0JeULl1X6iTn7VIkDeYX_X&index=33
Eron On Wed, Sep 27, 2017 at 9:12 AM, Stefan Richter <s.rich...@data-artisans.com > wrote: > Hi, > > from the top of my head, I cannot see why this should not be possible, > task managers just need to be able to connect to their job manager. > Unfortunately, I cannot give a real guarantee here because I am not that > deeply involved in this aspect of Flink. > > Best, > Stefan > > Am 27.09.2017 um 12:39 schrieb Rahul Raj <rahulrajms...@gmail.com>: > > Hi Stefan, > > I have a question in my mid out of curiosity.... Is it possible to run > flink application within docker container by using flink cluster set up on > host? > > Rahul Raj > > On 26 September 2017 at 17:29, Stefan Richter <s.rich...@data-artisans.com > > wrote: > >> Hi, >> >> if I correctly understood the approach outlined on github, you can start >> a standalone job manager and the task manager get the JM information either >> through the provided configuration or through Zookeeper. Take a look at the >> „running section“, e.g.: >> >> 1) „Via Mesos/Marathon: Start a standalone JobManager (you need to >> replace the flink_recovery_zookeeper_quorum variable with a valid >> setting for your cluster) [...]“ >> 2) „Via standalone Docker: Start a standalone JobManager (with host >> networking, binding on 127.0.0.1) […]“ >> >> Best, >> Stefan >> >> >> Am 26.09.2017 um 12:43 schrieb Rahul Raj <rahulrajms...@gmail.com>: >> >> Hi Stefan, >> >> Thanks a lot for your answer and sharing the link >> https://github.com/mesoshq/flink. I went through this and saw its >> spawning Jobmanager and taskmanager. Now I think, this should be happening. >> First JobManager will be started on flink cluster on one node, then task >> manager will be started on another node and both should be running in >> docker containers on different nodes. Now, my question is how flink's >> JobManager will get to know about the taskManagers as they are in in >> different docker containers on different nodes? Will it happen via Mesos? >> >> Can we use mesos-appmaster.sh instead which is already built in flink for >> deployment on mesos? >> >> Rahul Raj >> >> On 26 September 2017 at 15:32, Stefan Richter < >> s.rich...@data-artisans.com> wrote: >> >>> Hi, >>> >>> as in my answer to your previous mail, I suggest to take a look at >>> https://github.com/mesoshq/flink . Unfortunately, there is not yet a >>> lot documentation about the internals of how this works, so I am also >>> looping in Till who might know more about specific questions about how >>> things work together exactly. >>> >>> Best, >>> Stefan >>> >>> >>> Am 26.09.2017 um 09:21 schrieb Rahul Raj <rahulrajms...@gmail.com>: >>> >>> Currently I have a Flink Application Jar file running on Mesos cluster. >>> The flink application simply reads data from Kafka and put it to HDFS. >>> >>> Now we are planning to create a docker image to run this application >>> jar file inside docker containers on Mesos cluster via Marathon. >>> >>> Below are the questions that I am looking answers for: >>> >>> 1. While building the docker image, how do I include flink-1.3.2 set up >>> and my mesos config in flink? >>> >>> 2. How shall I run my existing flink application jar? >>> >>> 3. Will running my flink application jar on docker containers will run >>> it on mesos slaves on different docker containers? How docker , Flink, >>> mesos , Marathon will work together in my case? >>> >>> Rahul Raj >>> >>> >>> >> >> > >
