[GitHub] flink pull request: [doc] Fix wordcount example in YARN setup

2015-07-09 Thread jkirsch
GitHub user jkirsch opened a pull request: https://github.com/apache/flink/pull/897 [doc] Fix wordcount example in YARN setup Fix wrong wget call You can merge this pull request into a Git repository by running: $ git pull https://github.com/jkirsch/flink patch-1 Alternatively

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620828#comment-14620828 ] ASF GitHub Bot commented on FLINK-2008: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/895#issuecomment-120065921 Except for the issue mentioned by Gyula (the double commit of the head), this looks good. I would like to merge this later today or tomorrow. Could address Gyula's co

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620819#comment-14620819 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/896#issuecomment-120065197 This pattern seems to repeat in many places: ```java AccumulatorRegistry accumulatorRegistry = getEnvironment().getAccumulatorRegistry(); AccumulatorRegist

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34274983 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/accumulators/AccumulatorRegistry.java --- @@ -0,0 +1,140 @@ +/* + * Licensed to the

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620810#comment-14620810 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620801#comment-14620801 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34274318 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java --- @@ -172,13 +173,20 @@ /** The library cache, from wh

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620786#comment-14620786 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/896#issuecomment-120058469 The naming of the accumulators refers sometimes to "flink vs. user-defined", and sometimes to "internal vs. external". Can we make this consistent? I actually like th

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620780#comment-14620780 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-2341) Deadlock in SpilledSubpartitionViewAsyncIO

2015-07-09 Thread Ufuk Celebi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620782#comment-14620782 ] Ufuk Celebi commented on FLINK-2341: Thanks for the stacktrace. I will look into it so

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34273441 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java --- @@ -601,6 +636,11 @@ void markFinished() {

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620776#comment-14620776 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34273136 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java --- @@ -601,6 +636,11 @@ void markFinished() {

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34273000 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java --- @@ -131,6 +133,35 @@ private SerializedV

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620773#comment-14620773 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/896#issuecomment-120054333 The is a potential modification conflict: Drawing a snapshot for serialization and registering a new accumulator can lead to a ConcurrentModificationException in the

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620768#comment-14620768 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on the pu

[jira] [Commented] (FLINK-2340) Provide standalone mode for web interface of the JobManager

2015-07-09 Thread Ufuk Celebi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620767#comment-14620767 ] Ufuk Celebi commented on FLINK-2340: +1 I think separation is the way to go. > Provid

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34272308 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -1017,6 +1040,9 @@ object TaskManager {

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620762#comment-14620762 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34272101 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala --- @@ -306,99 +308,100 @@ extends Actor with ActorLogMessag

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620756#comment-14620756 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/896#issuecomment-120050532 It would be good if the accumulator update interval was configurable. --- If your project is set up for it, you can reply to this email and have your reply appear on

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620752#comment-14620752 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34271710 --- Diff: flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/wordcount/WordCount.java --- @@ -60,6 +60,7 @@ public static voi

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620750#comment-14620750 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620724#comment-14620724 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34270017 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/io/StreamingAbstractRecordReader.java --- @@

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620708#comment-14620708 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34269026 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java --- @@ -237,12 +246,13 @@ public Task(TaskDeploymentDescriptor tdd,

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620707#comment-14620707 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34268940 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java --- @@ -172,13 +173,20 @@ /** The library cache, from wh

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620697#comment-14620697 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34268803 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/RuntimeEnvironment.java --- @@ -212,19 +218,19 @@ public InputGate getInputGate

[jira] [Commented] (FLINK-2310) Add an Adamic-Adar Similarity example

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620693#comment-14620693 ] ASF GitHub Bot commented on FLINK-2310: --- Github user shghatge commented on the pull

[GitHub] flink pull request: [FLINK-2310] Add an Adamic Adar Similarity exa...

2015-07-09 Thread shghatge
Github user shghatge commented on the pull request: https://github.com/apache/flink/pull/892#issuecomment-120036933 Hello @vasia I would like to work on both versions of Adamic Adar. As the JIRA did not ask for an approximate version, it was suggested that I create another JIRA i

[GitHub] flink pull request: [FLINK-2310] Add an Adamic Adar Similarity exa...

2015-07-09 Thread shghatge
Github user shghatge commented on the pull request: https://github.com/apache/flink/pull/892#issuecomment-120033517 Hello @vasia I would like to work on both versions of Adamic Adar. As the JIRA did not ask for an approximate version, it was suggested that I create another JIRA i

[jira] [Commented] (FLINK-2310) Add an Adamic-Adar Similarity example

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620658#comment-14620658 ] ASF GitHub Bot commented on FLINK-2310: --- Github user shghatge commented on the pull

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620654#comment-14620654 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/896#discussion_r34266800 --- Diff: flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/wordcount/WordCount.java --- @@ -60,6 +60,7 @@ public static voi

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620651#comment-14620651 ] ASF GitHub Bot commented on FLINK-2292: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/896#issuecomment-120029960 Looks like this change breaks the YARN integration. The YARN WordCount no longer works. --- If your project is set up for it, you can reply to this email and have yo

[jira] [Commented] (FLINK-2329) Refactor RPCs from within the ExecutionGraph

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620642#comment-14620642 ] ASF GitHub Bot commented on FLINK-2329: --- Github user mxm commented on the pull reque

[GitHub] flink pull request: [FLINK-2329] [runtime] Introduces InstanceGate...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/893#issuecomment-120018025 Very sensible change in anticipation of the upcoming high availability changes. +1 --- If your project is set up for it, you can reply to this email and have your r

[jira] [Created] (FLINK-2341) Deadlock in SpilledSubpartitionViewAsyncIO

2015-07-09 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2341: --- Summary: Deadlock in SpilledSubpartitionViewAsyncIO Key: FLINK-2341 URL: https://issues.apache.org/jira/browse/FLINK-2341 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-2329) Refactor RPCs from within the ExecutionGraph

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620638#comment-14620638 ] ASF GitHub Bot commented on FLINK-2329: --- Github user mxm commented on a diff in the

[GitHub] flink pull request: [FLINK-2329] [runtime] Introduces InstanceGate...

2015-07-09 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/893#discussion_r34264492 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/instance/Instance.java --- @@ -43,8 +41,8 @@ /** The lock on which to synchronize allocation

[jira] [Commented] (FLINK-2288) Setup ZooKeeper for distributed coordination

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620634#comment-14620634 ] ASF GitHub Bot commented on FLINK-2288: --- Github user mxm commented on the pull reque

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-120015233 Alright, I've opened a JIRA for this: https://issues.apache.org/jira/browse/FLINK-2340 --- If your project is set up for it, you can reply to this email and have your reply

[jira] [Created] (FLINK-2340) Provide standalone mode for web interface of the JobManager

2015-07-09 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-2340: - Summary: Provide standalone mode for web interface of the JobManager Key: FLINK-2340 URL: https://issues.apache.org/jira/browse/FLINK-2340 Project: Flink

[jira] [Commented] (FLINK-2339) Prevent asynchronous checkpoint calls from overtaking each other

2015-07-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620625#comment-14620625 ] Stephan Ewen commented on FLINK-2339: - I have a patch coming up, currently testing on

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620615#comment-14620615 ] ASF GitHub Bot commented on FLINK-2008: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/895#discussion_r34262472 --- Diff: flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/Utils.java

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/895#discussion_r34262391 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamCheckpointingITCase.java --- @@ -53,6 +62,8 @@ @SuppressWarnings("serial")

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620613#comment-14620613 ] ASF GitHub Bot commented on FLINK-2008: --- Github user rmetzger commented on a diff in

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620611#comment-14620611 ] ASF GitHub Bot commented on FLINK-2008: --- Github user rmetzger commented on a diff in

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/895#discussion_r34262145 --- Diff: flink-tests/src/test/resources/log4j-test.properties --- @@ -18,7 +18,7 @@ # Set root logger level to OFF to not flood build logs # s

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620610#comment-14620610 ] ASF GitHub Bot commented on FLINK-2008: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/895#discussion_r34262141 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamCheckpointingITCase.java --- @@ -53,6 +62,8 @@ @SuppressWarnings("seria

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620609#comment-14620609 ] ASF GitHub Bot commented on FLINK-2008: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620608#comment-14620608 ] ASF GitHub Bot commented on FLINK-2008: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/895#discussion_r34262084 --- Diff: flink-tests/src/test/resources/log4j-test.properties --- @@ -18,7 +18,7 @@ # Set root logger level to OFF to not flood build logs

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/895#discussion_r34261966 --- Diff: docs/apis/streaming_guide.md --- @@ -1300,6 +1300,9 @@ Another way of exposing user defined operator state for the Flink runtime for ch

[jira] [Commented] (FLINK-2288) Setup ZooKeeper for distributed coordination

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620572#comment-14620572 ] ASF GitHub Bot commented on FLINK-2288: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-12744 The web interface is, modulo some object which are not serializable, already independent of the JobManager. It should not be a big problem to only have one web

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-12117 Yes that makes sense. So the user will always have to connect to the web interface of the leading job manager, right? We could only circumvent that by separating the web inte

[jira] [Commented] (FLINK-2288) Setup ZooKeeper for distributed coordination

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620568#comment-14620568 ] ASF GitHub Bot commented on FLINK-2288: --- Github user mxm commented on the pull reque

[jira] [Commented] (FLINK-2288) Setup ZooKeeper for distributed coordination

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620550#comment-14620550 ] ASF GitHub Bot commented on FLINK-2288: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119996182 In HA mode, JobManagers start with a random free port. That is fine, because no one connects to them based on a config value, but only based on ZooKeeper entries. -

[GitHub] flink pull request: [wip] [FLINK-2288] [FLINK-2302] Setup ZooKeepe...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/886#issuecomment-119988165 Did we find a solution for the random port problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[jira] [Commented] (FLINK-2288) Setup ZooKeeper for distributed coordination

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620540#comment-14620540 ] ASF GitHub Bot commented on FLINK-2288: --- Github user mxm commented on the pull reque

[jira] [Created] (FLINK-2339) Prevent asynchronous checkpoint calls from overtaking each other

2015-07-09 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2339: --- Summary: Prevent asynchronous checkpoint calls from overtaking each other Key: FLINK-2339 URL: https://issues.apache.org/jira/browse/FLINK-2339 Project: Flink

[GitHub] flink pull request: [FLINK-2235] fix calculation of free memory fo...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/859#issuecomment-119987554 1/4 of the physical memory seems sensible and indeed returns almost the same memory size as the maximum memory setting on my machine. I've adapted the pull request. --- If

[jira] [Commented] (FLINK-2235) Local Flink cluster allocates too much memory

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620538#comment-14620538 ] ASF GitHub Bot commented on FLINK-2235: --- Github user mxm commented on the pull reque

[GitHub] flink pull request: [FLINK-2292][FLINK-1573] add live per-task acc...

2015-07-09 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/896 [FLINK-2292][FLINK-1573] add live per-task accumulators This refactors the accumulators to accumulate per task execution. The accumulators are reported from the task managers periodically to the job

[jira] [Commented] (FLINK-2292) Report accumulators periodically while job is running

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620523#comment-14620523 ] ASF GitHub Bot commented on FLINK-2292: --- GitHub user mxm opened a pull request:

[GitHub] flink pull request: [FLINK-2008][FLINK-2296] Fix checkpoint commit...

2015-07-09 Thread rmetzger
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/895 [FLINK-2008][FLINK-2296] Fix checkpoint committing & KafkaITCase [wip] You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink flink2008

[jira] [Commented] (FLINK-2008) PersistentKafkaSource is sometimes emitting tuples multiple times

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620459#comment-14620459 ] ASF GitHub Bot commented on FLINK-2008: --- GitHub user rmetzger opened a pull request:

[jira] [Commented] (FLINK-2162) Implement adaptive learning rate strategies for SGD

2015-07-09 Thread Till Rohrmann (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620427#comment-14620427 ] Till Rohrmann commented on FLINK-2162: -- Great to hear [~ventura] :-) I've assigned yo

[jira] [Updated] (FLINK-2162) Implement adaptive learning rate strategies for SGD

2015-07-09 Thread Till Rohrmann (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann updated FLINK-2162: - Assignee: Ventura Del Monte > Implement adaptive learning rate strategies for SGD > --

[jira] [Updated] (FLINK-2338) Shut down "Storm Topologies" cleanly

2015-07-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated FLINK-2338: --- Summary: Shut down "Storm Topologies" cleanly (was: Shut down "Storm Topologies" clenaly) >

[jira] [Commented] (FLINK-2305) Add documenation about Storm compatibility layer

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620418#comment-14620418 ] ASF GitHub Bot commented on FLINK-2305: --- Github user mjsax commented on the pull req

[GitHub] flink pull request: [FLINK-2305] Add documenation about Storm comp...

2015-07-09 Thread mjsax
Github user mjsax commented on the pull request: https://github.com/apache/flink/pull/884#issuecomment-119943363 I just created two JIRAs: - SLF4J: https://issues.apache.org/jira/browse/FLINK-2337 - JobExecutionException: https://issues.apache.org/jira/browse/FLINK-2338

[jira] [Created] (FLINK-2338) Shut down "Storm Topologies" clenaly

2015-07-09 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2338: -- Summary: Shut down "Storm Topologies" clenaly Key: FLINK-2338 URL: https://issues.apache.org/jira/browse/FLINK-2338 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-2337) Multiple SLF4J bindings using Storm compatibility layer

2015-07-09 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2337: -- Summary: Multiple SLF4J bindings using Storm compatibility layer Key: FLINK-2337 URL: https://issues.apache.org/jira/browse/FLINK-2337 Project: Flink Iss

[jira] [Commented] (FLINK-2162) Implement adaptive learning rate strategies for SGD

2015-07-09 Thread Ventura Del Monte (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620398#comment-14620398 ] Ventura Del Monte commented on FLINK-2162: -- Hi, I would like to contribute to Fli

[jira] [Commented] (FLINK-2336) ArrayIndexOufOBoundsException in TypeExtractor when mapping

2015-07-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620389#comment-14620389 ] Stephan Ewen commented on FLINK-2336: - Can you post a simpifies data type that reprodu

[jira] [Commented] (FLINK-2336) ArrayIndexOufOBoundsException in TypeExtractor when mapping

2015-07-09 Thread William Saar (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620345#comment-14620345 ] William Saar commented on FLINK-2336: - When replacing the lambda with a MapFunction in

[jira] [Created] (FLINK-2336) ArrayIndexOufOBoundsException in TypeExtractor when mapping

2015-07-09 Thread William Saar (JIRA)
William Saar created FLINK-2336: --- Summary: ArrayIndexOufOBoundsException in TypeExtractor when mapping Key: FLINK-2336 URL: https://issues.apache.org/jira/browse/FLINK-2336 Project: Flink Issu

[jira] [Created] (FLINK-2335) Rework iteration construction in StreamGraph

2015-07-09 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-2335: - Summary: Rework iteration construction in StreamGraph Key: FLINK-2335 URL: https://issues.apache.org/jira/browse/FLINK-2335 Project: Flink Issue Type: Improvement

[jira] [Commented] (FLINK-2235) Local Flink cluster allocates too much memory

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620197#comment-14620197 ] ASF GitHub Bot commented on FLINK-2235: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2235] fix calculation of free memory fo...

2015-07-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/859#issuecomment-119891011 Okay, from skimming over some Oracle docs, it seems the default max heap is 1/4 of the physical memory. Let's use that. --- If your project is set up for it, you can

[GitHub] flink pull request: [FLINK-2235] fix calculation of free memory fo...

2015-07-09 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/859#issuecomment-119883278 Typically, programs can allocate as much memory as they like. We only take a fraction of the free physical memory for the manged memory. We could also take only half of the p

[jira] [Commented] (FLINK-2235) Local Flink cluster allocates too much memory

2015-07-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620163#comment-14620163 ] ASF GitHub Bot commented on FLINK-2235: --- Github user mxm commented on the pull reque

[jira] [Closed] (FLINK-2280) GenericTypeComparator.compare() does not respect ascending flag

2015-07-09 Thread Fabian Hueske (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Hueske closed FLINK-2280. Resolution: Fixed Fixed with 590c6d0644e59d0bc3b5b6f4869f91bc38864232 > GenericTypeComparator.compa

[jira] [Created] (FLINK-2334) IOException: Channel to path could not be opened

2015-07-09 Thread David Heller (JIRA)
David Heller created FLINK-2334: --- Summary: IOException: Channel to path could not be opened Key: FLINK-2334 URL: https://issues.apache.org/jira/browse/FLINK-2334 Project: Flink Issue Type: Bug

[jira] [Updated] (FLINK-2328) Applying more than one transformation on an IterativeDataStream fails

2015-07-09 Thread Gyula Fora (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora updated FLINK-2328: -- Component/s: Streaming > Applying more than one transformation on an IterativeDataStream fails > ---

  1   2   >