Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-22 Thread Deepak Jha
Hi Maximilian, Thanks for the email and looking into the issue. I'm using Scala 2.11 so it sounds perfect to me... I will be more than happy to test it out. On Tue, Mar 22, 2016 at 2:48 AM, Maximilian Michels wrote: > Hi Deepak, > > We have looked further into this and have a pretty easy fix. Ho

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-22 Thread Maximilian Michels
Hi Deepak, We have looked further into this and have a pretty easy fix. However, it will only work with Flink's Scala 2.11 version because newer versions of the Akka library are incompatible with Scala 2.10 (Flink's default Scala version). Would that be a viable option for you? We're currently di

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-14 Thread Deepak Jha
Hi Maximilian, Thanks for your response. I will wait for the update. On Monday, March 14, 2016, Maximilian Michels wrote: > Hi Deepak, > > We'll look more into this problem this week. Until now we considered it a > configuration issue if the bind address was not externally reachable. > However,

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-14 Thread Maximilian Michels
Hi Deepak, We'll look more into this problem this week. Until now we considered it a configuration issue if the bind address was not externally reachable. However, one might not always have the possibility to change this network configuration. Looking further, it is actually possible to let the b

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-13 Thread Deepak Jha
Hi Stephan & Ufuk, Thanks for your response. Yes there is a way in which you can run docker (net = host mode) in which guest machine's network stack gets shared by docker container. Unfortunately its not supported by AWS ECS. I do have one more question for you. Can you guys please explain me wha

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-11 Thread Stephan Ewen
Hi Deepak! We can currently not split the bind address and advertised address, because the Akka library only accepts packages sent explicitly to the bind address (not sure why Akka has this artificial limitation, but it is there). Can you bridge the container IP address to be visible from the out

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-11 Thread Ufuk Celebi
Hey Deepak! Your description of Flink's behaviour is correct. To summarize: # Host Address If you specify a host address as an argument to the JVM (via jobmanager.sh or the start-cluster.sh scripts) then that one is used. If you don't, it falls back to the value configured in flink-conf.yaml (wh

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-10 Thread Deepak Jha
Hi Stephan, I am able to figure out the issue... Here is my explanation.. As I've said, I'm trying to setup Flink HA cluster in docker containers managed by Amazon ECS. I've a remote zookeeper cluster running in AWS. There are few issues when we deploy it using docker --- Flink uses *jobmanager.r

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-10 Thread Stephan Ewen
Hi! Is it possible that the docker container config forbids to open ports? Flink will try to open some ports and needs the OS or container to permit that. Greetings, Stephan On Thu, Mar 10, 2016 at 6:27 PM, Deepak Jha wrote: > Hi Stephan, > I tried 0.10.2 as well still running into the same i

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-10 Thread Deepak Jha
Hi Stephan, I tried 0.10.2 as well still running into the same issue. On Thursday, March 10, 2016, Deepak Jha wrote: > Yes. Flink 1.0.0 > > On Thursday, March 10, 2016, Stephan Ewen > wrote: > >> Hi! >> >> Is this Flink 1.0.0 ? >> >> Stephan >> >> >> On Thu, Mar 10, 2016 at 6:02 AM, Deepak Jha

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-10 Thread Deepak Jha
Yes. Flink 1.0.0 On Thursday, March 10, 2016, Stephan Ewen wrote: > Hi! > > Is this Flink 1.0.0 ? > > Stephan > > > On Thu, Mar 10, 2016 at 6:02 AM, Deepak Jha > wrote: > > > Hi All, > > > > I'm trying to setup Flink 1.0.0 cluster on Docker (separate containers > for > > jobmanager and taskmana

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-10 Thread Stephan Ewen
Hi! Is this Flink 1.0.0 ? Stephan On Thu, Mar 10, 2016 at 6:02 AM, Deepak Jha wrote: > Hi All, > > I'm trying to setup Flink 1.0.0 cluster on Docker (separate containers for > jobmanager and taskmanager) inside AWS (Using AWS ECS service). I tested it > locally and its working fine but on AWS