Hi, from the top of my head, I cannot see why this should not be possible, task managers just need to be able to connect to their job manager. Unfortunately, I cannot give a real guarantee here because I am not that deeply involved in this aspect of Flink.
Best, Stefan > Am 27.09.2017 um 12:39 schrieb Rahul Raj <rahulrajms...@gmail.com>: > > Hi Stefan, > > I have a question in my mid out of curiosity.... Is it possible to run flink > application within docker container by using flink cluster set up on host? > > Rahul Raj > > On 26 September 2017 at 17:29, Stefan Richter <s.rich...@data-artisans.com > <mailto:s.rich...@data-artisans.com>> wrote: > Hi, > > if I correctly understood the approach outlined on github, you can start a > standalone job manager and the task manager get the JM information either > through the provided configuration or through Zookeeper. Take a look at the > „running section“, e.g.: > > 1) „Via Mesos/Marathon: Start a standalone JobManager (you need to replace > the flink_recovery_zookeeper_quorum variable with a valid setting for your > cluster) [...]“ > 2) „Via standalone Docker: Start a standalone JobManager (with host > networking, binding on 127.0.0.1) […]“ > > Best, > Stefan > > >> Am 26.09.2017 um 12:43 schrieb Rahul Raj <rahulrajms...@gmail.com >> <mailto:rahulrajms...@gmail.com>>: >> >> Hi Stefan, >> >> Thanks a lot for your answer and sharing the link >> https://github.com/mesoshq/flink <https://github.com/mesoshq/flink>. I went >> through this and saw its spawning Jobmanager and taskmanager. Now I think, >> this should be happening. First JobManager will be started on flink cluster >> on one node, then task manager will be started on another node and both >> should be running in docker containers on different nodes. Now, my question >> is how flink's JobManager will get to know about the taskManagers as they >> are in in different docker containers on different nodes? Will it happen via >> Mesos? >> >> Can we use mesos-appmaster.sh instead which is already built in flink for >> deployment on mesos? >> >> Rahul Raj >> >> On 26 September 2017 at 15:32, Stefan Richter <s.rich...@data-artisans.com >> <mailto:s.rich...@data-artisans.com>> wrote: >> Hi, >> >> as in my answer to your previous mail, I suggest to take a look at >> https://github.com/mesoshq/flink <https://github.com/mesoshq/flink> . >> Unfortunately, there is not yet a lot documentation about the internals of >> how this works, so I am also looping in Till who might know more about >> specific questions about how things work together exactly. >> >> Best, >> Stefan >> >>> Am 26.09.2017 um 09:21 schrieb Rahul Raj <rahulrajms...@gmail.com >>> <mailto:rahulrajms...@gmail.com>>: >>> >>> Currently I have a Flink Application Jar file running on Mesos cluster. The >>> flink application simply reads data from Kafka and put it to HDFS. >>> >>> Now we are planning to create a docker image to run this application jar >>> file inside docker containers on Mesos cluster via Marathon. >>> >>> Below are the questions that I am looking answers for: >>> >>> 1. While building the docker image, how do I include flink-1.3.2 set up and >>> my mesos config in flink? >>> >>> 2. How shall I run my existing flink application jar? >>> >>> 3. Will running my flink application jar on docker containers will run it >>> on mesos slaves on different docker containers? How docker , Flink, mesos , >>> Marathon will work together in my case? >>> >>> Rahul Raj >> >> > >
