[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-120015233 Alright, I've opened a JIRA for this: https://issues.apache.org/jira/browse/FLINK-2340 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-12744 The web interface is, modulo some object which are not serializable, already independent of the JobManager. It should not be a big problem to only have one web

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-12117 Yes that makes sense. So the user will always have to connect to the web interface of the leading job manager, right? We could only circumvent that by separating the web inte

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119996182 In HA mode, JobManagers start with a random free port. That is fine, because no one connects to them based on a config value, but only based on ZooKeeper entries. -

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119988165 Did we find a solution for the random port problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/886 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-08 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119553437 We need to find a solution for the webfrontend, though. Starting it on random ports is not a real solution. Also the user that wants to see progress needs to connect

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-08 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119553291 BTW: It is beautiful how easy it is to start multiple TaskManagers and JobManagers on one machine now :-) --- If your project is set up for it, you can reply to this

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-08 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119529122 I will address the comments and merge this... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34050754 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -993,9 +1000,34 @@ object JobManager { conf

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34047058 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -993,9 +1000,34 @@ object JobManager { confi

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34045870 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -993,9 +1000,34 @@ object JobManager { conf

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119202663 Looks like a good piece of work! Can we actually get into ZooKeeper version conflicts here? For example, the Kafka connector needs a certain Zookeeper version

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34038430 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -1435,6 +1435,11 @@ object TaskManager { //

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34038403 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -993,9 +1000,34 @@ object JobManager { confi

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34038257 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -993,9 +1000,34 @@ object JobManager { confi

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34037018 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/FlinkZooKeeperQuorumPeer.java --- @@ -0,0 +1,227 @@ +/* + * Licensed to t

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34036874 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/FlinkZooKeeperQuorumPeer.java --- @@ -0,0 +1,227 @@ +/* + * Licensed to t

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-07 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/886#discussion_r34035723 --- Diff: docs/setup/jobmanager_high_availability.md --- @@ -0,0 +1,121 @@ +--- +title: "JobManager High Availability (HA)" +--- + +

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-03 Thread uce
GitHub user uce opened a pull request: https://github.com/apache/flink/pull/886 [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeeper for distributed coordination - FLINK-2288: Setup ZooKeeper for distributed coordination * Add FlinkZooKeeperQuorumPeer to wrap ZooKeeper's quorum peer