[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528764#comment-15528764 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on the issue

[GitHub] flink issue #2542: [FLINK-4613] Extend ALS to handle implicit feedback datas...

2016-09-28 Thread thvasilo
Github user thvasilo commented on the issue: https://github.com/apache/flink/pull/2542 Hello @gaborhermann thank you for your contribution! Are the numbers here non-zero entries in a matrix? If that is the case do you think it would be possible to test this on some larger scal

[jira] [Commented] (FLINK-4632) when yarn nodemanager lost, flink hung

2016-09-28 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528781#comment-15528781 ] 刘喆 commented on FLINK-4632: --- I can't reproduce it now. I only save the TaskManager' log as the b

[GitHub] flink pull request #2542: [FLINK-4613] Extend ALS to handle implicit feedbac...

2016-09-28 Thread thvasilo
Github user thvasilo commented on a diff in the pull request: https://github.com/apache/flink/pull/2542#discussion_r80861067 --- Diff: flink-libraries/flink-ml/src/main/scala/org/apache/flink/ml/recommendation/ALS.scala --- @@ -581,6 +637,16 @@ object ALS { val userXy

[GitHub] flink pull request #2542: [FLINK-4613] Extend ALS to handle implicit feedbac...

2016-09-28 Thread thvasilo
Github user thvasilo commented on a diff in the pull request: https://github.com/apache/flink/pull/2542#discussion_r80862018 --- Diff: flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/recommendation/ImplicitALSTest.scala --- @@ -0,0 +1,171 @@ +/* + * Licensed to

[GitHub] flink pull request #2542: [FLINK-4613] Extend ALS to handle implicit feedbac...

2016-09-28 Thread thvasilo
Github user thvasilo commented on a diff in the pull request: https://github.com/apache/flink/pull/2542#discussion_r80860057 --- Diff: docs/dev/libs/ml/als.md --- @@ -49,6 +49,21 @@ By applying this step alternately to the matrices $U$ and $V$, we can iterativel The matr

[GitHub] flink pull request #2542: [FLINK-4613] Extend ALS to handle implicit feedbac...

2016-09-28 Thread thvasilo
Github user thvasilo commented on a diff in the pull request: https://github.com/apache/flink/pull/2542#discussion_r80860320 --- Diff: docs/dev/libs/ml/als.md --- @@ -99,6 +114,26 @@ The alternating least squares implementation can be controlled by the following

[GitHub] flink pull request #2542: [FLINK-4613] Extend ALS to handle implicit feedbac...

2016-09-28 Thread thvasilo
Github user thvasilo commented on a diff in the pull request: https://github.com/apache/flink/pull/2542#discussion_r80860723 --- Diff: docs/dev/libs/ml/als.md --- @@ -99,6 +114,26 @@ The alternating least squares implementation can be controlled by the following

[GitHub] flink pull request #2542: [FLINK-4613] Extend ALS to handle implicit feedbac...

2016-09-28 Thread thvasilo
Github user thvasilo commented on a diff in the pull request: https://github.com/apache/flink/pull/2542#discussion_r80862241 --- Diff: flink-libraries/flink-ml/src/test/scala/org/apache/flink/ml/recommendation/ImplicitALSTest.scala --- @@ -0,0 +1,171 @@ +/* + * Licensed to

[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528811#comment-15528811 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on a diff in

[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528815#comment-15528815 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on a diff in

[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528814#comment-15528814 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on a diff in

[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528816#comment-15528816 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on a diff in

[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528813#comment-15528813 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on a diff in

[jira] [Commented] (FLINK-4613) Extend ALS to handle implicit feedback datasets

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528812#comment-15528812 ] ASF GitHub Bot commented on FLINK-4613: --- Github user thvasilo commented on a diff in

[jira] [Commented] (FLINK-4632) when yarn nodemanager lost, flink hung

2016-09-28 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528833#comment-15528833 ] 刘喆 commented on FLINK-4632: --- I think it is related to checkpoint. When I use checkpoint with 'ex

[jira] [Comment Edited] (FLINK-4632) when yarn nodemanager lost, flink hung

2016-09-28 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528833#comment-15528833 ] 刘喆 edited comment on FLINK-4632 at 9/28/16 8:13 AM: I think it is rela

[jira] [Comment Edited] (FLINK-4632) when yarn nodemanager lost, flink hung

2016-09-28 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528833#comment-15528833 ] 刘喆 edited comment on FLINK-4632 at 9/28/16 8:25 AM: I think it is rela

[GitHub] flink pull request #2550: [FLINK-4657] Implement HighAvailabilityServices ba...

2016-09-28 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/2550#discussion_r80869838 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java --- @@ -467,6 +487,128 @@ public void registerAtResourceManager(

[jira] [Commented] (FLINK-4657) Implement HighAvailabilityServices based on zookeeper

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528958#comment-15528958 ] ASF GitHub Bot commented on FLINK-4657: --- Github user tillrohrmann commented on a dif

[jira] [Commented] (FLINK-4068) Move constant computations out of code-generated `flatMap` functions.

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528968#comment-15528968 ] ASF GitHub Bot commented on FLINK-4068: --- Github user twalthr commented on the issue:

[GitHub] flink issue #2560: [FLINK-4068] [table] Move constant computations out of co...

2016-09-28 Thread twalthr
Github user twalthr commented on the issue: https://github.com/apache/flink/pull/2560 I will shepherd this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] flink issue #2461: [FLINK-4505][Cluster Management] Implement TaskManagerFac...

2016-09-28 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/2461 I've merged your PR @wangzhijiang999. Thanks for your work :-) You can close this PR now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[jira] [Resolved] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-28 Thread Till Rohrmann (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann resolved FLINK-4505. -- Resolution: Fixed Added via 5513fe61dc8699a617c163ee2a555e43dc6422e4 > Implement TaskManagerRun

[jira] [Commented] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529008#comment-15529008 ] ASF GitHub Bot commented on FLINK-4505: --- Github user tillrohrmann commented on the i

[GitHub] flink pull request #2550: [FLINK-4657] Implement HighAvailabilityServices ba...

2016-09-28 Thread KurtYoung
Github user KurtYoung commented on a diff in the pull request: https://github.com/apache/flink/pull/2550#discussion_r80874290 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/JobMaster.java --- @@ -467,6 +487,128 @@ public void registerAtResourceManager(fin

[jira] [Commented] (FLINK-4657) Implement HighAvailabilityServices based on zookeeper

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529018#comment-15529018 ] ASF GitHub Bot commented on FLINK-4657: --- Github user KurtYoung commented on a diff i

[jira] [Commented] (FLINK-4506) CsvOutputFormat defaults allowNullValues to false, even though doc and declaration says true

2016-09-28 Thread Kirill Morozov (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529066#comment-15529066 ] Kirill Morozov commented on FLINK-4506: --- Rollbacked to old behavior, documentation f

[jira] [Closed] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-28 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang closed FLINK-4505. > Implement TaskManagerRunner to construct related components for TaskManager >

[GitHub] flink pull request #2461: [FLINK-4505][Cluster Management] Implement TaskMan...

2016-09-28 Thread wangzhijiang999
Github user wangzhijiang999 closed the pull request at: https://github.com/apache/flink/pull/2461 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[jira] [Commented] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529111#comment-15529111 ] ASF GitHub Bot commented on FLINK-4505: --- Github user wangzhijiang999 closed the pull

[GitHub] flink issue #2461: [FLINK-4505][Cluster Management] Implement TaskManagerFac...

2016-09-28 Thread wangzhijiang999
Github user wangzhijiang999 commented on the issue: https://github.com/apache/flink/pull/2461 @tillrohrmann , thank you for merging and help. If there are any following works to do related with TaskManager, you can assign to me and I am willing to do. --- If your project is set up f

[jira] [Commented] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529109#comment-15529109 ] ASF GitHub Bot commented on FLINK-4505: --- Github user wangzhijiang999 commented on th

[jira] [Commented] (FLINK-4563) [metrics] scope caching not adjusted for multiple reporters

2016-09-28 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529170#comment-15529170 ] Chesnay Schepler commented on FLINK-4563: - yes, those tests should pass. additiona

[jira] [Commented] (FLINK-4573) Potential resource leak due to unclosed RandomAccessFile in TaskManagerLogHandler

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529192#comment-15529192 ] ASF GitHub Bot commented on FLINK-4573: --- Github user zentol commented on a diff in t

[GitHub] flink pull request #2556: [FLINK-4573] Fix potential resource leak due to un...

2016-09-28 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/2556#discussion_r80885273 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/TaskManagerLogHandler.java --- @@ -210,7 +210,17 @@ public void onSucce

[jira] [Commented] (FLINK-4573) Potential resource leak due to unclosed RandomAccessFile in TaskManagerLogHandler

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529194#comment-15529194 ] ASF GitHub Bot commented on FLINK-4573: --- Github user zentol commented on a diff in t

[GitHub] flink pull request #2556: [FLINK-4573] Fix potential resource leak due to un...

2016-09-28 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/2556#discussion_r80885477 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/TaskManagerLogHandler.java --- @@ -210,7 +210,17 @@ public void onSucce

[GitHub] flink pull request #2517: [FLINK-4564] [metrics] Delimiter should be configu...

2016-09-28 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/2517#discussion_r80886088 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistry.java --- @@ -216,17 +244,20 @@ public ScopeFormats getScopeFormats() {

[GitHub] flink pull request #2517: [FLINK-4564] [metrics] Delimiter should be configu...

2016-09-28 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/2517#discussion_r80886970 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistry.java --- @@ -216,17 +244,20 @@ public ScopeFormats getScopeFormats() {

[jira] [Commented] (FLINK-4564) [metrics] Delimiter should be configured per reporter

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529216#comment-15529216 ] ASF GitHub Bot commented on FLINK-4564: --- Github user zentol commented on a diff in t

[jira] [Commented] (FLINK-4564) [metrics] Delimiter should be configured per reporter

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529215#comment-15529215 ] ASF GitHub Bot commented on FLINK-4564: --- Github user zentol commented on a diff in t

[GitHub] flink pull request #2477: [FLINK-4506] CsvOutputFormat defaults allowNullVal...

2016-09-28 Thread zentol
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/2477#discussion_r80887299 --- Diff: flink-java/src/test/java/org/apache/flink/api/java/io/CsvOutputFormatTest.java --- @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (FLINK-4506) CsvOutputFormat defaults allowNullValues to false, even though doc and declaration says true

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529220#comment-15529220 ] ASF GitHub Bot commented on FLINK-4506: --- Github user zentol commented on a diff in t

[jira] [Created] (FLINK-4703) RpcCompletenessTest: Add support for type arguments and subclasss

2016-09-28 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-4703: - Summary: RpcCompletenessTest: Add support for type arguments and subclasss Key: FLINK-4703 URL: https://issues.apache.org/jira/browse/FLINK-4703 Project: Fl

[jira] [Updated] (FLINK-4703) RpcCompletenessTest: Add support for type arguments and subclasses

2016-09-28 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated FLINK-4703: -- Summary: RpcCompletenessTest: Add support for type arguments and subclasses (was: RpcCo

[GitHub] flink issue #2527: [FLINK-4624] Allow for null values in Graph Summarization

2016-09-28 Thread s1ck
Github user s1ck commented on the issue: https://github.com/apache/flink/pull/2527 @greghogan since I addressed all your comments, is there anything left to do that prevents this from merging? --- If your project is set up for it, you can reply to this email and have your reply appea

[jira] [Commented] (FLINK-4624) Gelly's summarization algorithm cannot deal with null vertex group values

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529234#comment-15529234 ] ASF GitHub Bot commented on FLINK-4624: --- Github user s1ck commented on the issue:

[GitHub] flink pull request #2561: [FLINK-4703] RpcCompletenessTest: Add support for ...

2016-09-28 Thread mxm
GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/2561 [FLINK-4703] RpcCompletenessTest: Add support for type arguments and subclasses This is a prerequisite for merging #2540 which relies on the class hierarchy features. You can merge this pull request i

[jira] [Commented] (FLINK-4703) RpcCompletenessTest: Add support for type arguments and subclasses

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529239#comment-15529239 ] ASF GitHub Bot commented on FLINK-4703: --- GitHub user mxm opened a pull request:

[jira] [Comment Edited] (FLINK-4632) when yarn nodemanager lost, flink hung

2016-09-28 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528833#comment-15528833 ] 刘喆 edited comment on FLINK-4632 at 9/28/16 11:00 AM: - I think it is re

[GitHub] flink issue #2559: [FLINK-4702] [kafka connector] Commit offets to Kafka asy...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2559 @robert and @tzulitai What is your take on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529398#comment-15529398 ] ASF GitHub Bot commented on FLINK-4702: --- Github user StephanEwen commented on the is

[GitHub] flink pull request #2477: [FLINK-4506] CsvOutputFormat defaults allowNullVal...

2016-09-28 Thread kirill-morozov-epam
Github user kirill-morozov-epam commented on a diff in the pull request: https://github.com/apache/flink/pull/2477#discussion_r80901404 --- Diff: flink-java/src/test/java/org/apache/flink/api/java/io/CsvOutputFormatTest.java --- @@ -0,0 +1,63 @@ +/* + * Licensed to the Apa

[jira] [Commented] (FLINK-4506) CsvOutputFormat defaults allowNullValues to false, even though doc and declaration says true

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529420#comment-15529420 ] ASF GitHub Bot commented on FLINK-4506: --- Github user kirill-morozov-epam commented o

[GitHub] flink pull request #2562: [FLINK-4691] [table] Add group-windows for streami...

2016-09-28 Thread twalthr
GitHub user twalthr opened a pull request: https://github.com/apache/flink/pull/2562 [FLINK-4691] [table] Add group-windows for streaming tables This PR implements Tumble, Slide, Session group-windows for streaming tables as described in FLIP-11. It adds API, validation, logical re

[jira] [Commented] (FLINK-4691) Add group-windows for streaming tables

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529423#comment-15529423 ] ASF GitHub Bot commented on FLINK-4691: --- GitHub user twalthr opened a pull request:

[GitHub] flink issue #2559: [FLINK-4702] [kafka connector] Commit offets to Kafka asy...

2016-09-28 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2559 Just had a look at the API of `commitAsync`, and it seems like the committed offsets back to Kafka through this API (likewise for `commitSync`) need to be `lastProcessedMessageOffset + 1` ([https:/

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529448#comment-15529448 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on the issue

[GitHub] flink pull request #2559: [FLINK-4702] [kafka connector] Commit offets to Ka...

2016-09-28 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/2559#discussion_r80903481 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java --- @@

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529450#comment-15529450 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on a diff in

[GitHub] flink issue #2559: [FLINK-4702] [kafka connector] Commit offets to Kafka asy...

2016-09-28 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2559 Btw, just curious, does 0.8 Kafka connector have the same issue with sync committing? I haven't looked into the code for this, but just wondering if we need a ticket for 0.8 too. --- If your proje

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529455#comment-15529455 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on the issue

[jira] [Assigned] (FLINK-4406) Implement job master registration at resource manager

2016-09-28 Thread Kurt Young (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young reassigned FLINK-4406: - Assignee: Kurt Young (was: zhuhaifeng) > Implement job master registration at resource manager >

[jira] [Commented] (FLINK-4530) Generalize TaskExecutorToResourceManagerConnection to be reusable

2016-09-28 Thread Kurt Young (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529460#comment-15529460 ] Kurt Young commented on FLINK-4530: --- fixed by b955465ff2f230da0ecd195d7d0e8312fdf0578e

[jira] [Resolved] (FLINK-4530) Generalize TaskExecutorToResourceManagerConnection to be reusable

2016-09-28 Thread Kurt Young (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young resolved FLINK-4530. --- Resolution: Fixed > Generalize TaskExecutorToResourceManagerConnection to be reusable > --

[jira] [Commented] (FLINK-4618) FlinkKafkaConsumer09 should start from the next record on startup from offsets in Kafka

2016-09-28 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529466#comment-15529466 ] Tzu-Li (Gordon) Tai commented on FLINK-4618: Hi [~melmoth], I just had a look

[GitHub] flink issue #2559: [FLINK-4702] [kafka connector] Commit offets to Kafka asy...

2016-09-28 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2559 @StephanEwen I think you've tagged the wrong Github ID for Robert ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529483#comment-15529483 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on the issue

[jira] [Comment Edited] (FLINK-4618) FlinkKafkaConsumer09 should start from the next record on startup from offsets in Kafka

2016-09-28 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529466#comment-15529466 ] Tzu-Li (Gordon) Tai edited comment on FLINK-4618 at 9/28/16 12:43 PM: --

[GitHub] flink issue #2460: [FLINK-4562] table examples make an divided module in fli...

2016-09-28 Thread twalthr
Github user twalthr commented on the issue: https://github.com/apache/flink/pull/2460 +1 I will shepherd this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] flink pull request #2559: [FLINK-4702] [kafka connector] Commit offets to Ka...

2016-09-28 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/2559#discussion_r80906814 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java --- @@

[jira] [Commented] (FLINK-4562) table examples make an divided module in flink-examples

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529490#comment-15529490 ] ASF GitHub Bot commented on FLINK-4562: --- Github user twalthr commented on the issue:

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529493#comment-15529493 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on a diff in

[GitHub] flink pull request #2559: [FLINK-4702] [kafka connector] Commit offets to Ka...

2016-09-28 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/2559#discussion_r80907326 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java --- @@

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529502#comment-15529502 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on a diff in

[GitHub] flink issue #2559: [FLINK-4702] [kafka connector] Commit offets to Kafka asy...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2559 Thanks @tzulitai for looking at this. I will leave the offset then as it is (fixed via followup) and The Kafka 0.8 connector needs a similar change. This here is encountered by a user,

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529504#comment-15529504 ] ASF GitHub Bot commented on FLINK-4702: --- Github user StephanEwen commented on the is

[GitHub] flink pull request #2559: [FLINK-4702] [kafka connector] Commit offets to Ka...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2559#discussion_r80909904 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java ---

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529532#comment-15529532 ] ASF GitHub Bot commented on FLINK-4702: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request #2546: [FLINK-4329] Fix Streaming File Source Timestamps/...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2546#discussion_r80901203 --- Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/AggregatingAlignedProcessingTimeWindowOperatorTest.java -

[GitHub] flink pull request #2546: [FLINK-4329] Fix Streaming File Source Timestamps/...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2546#discussion_r80901399 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/DefaultTimeServiceProvider.java --- @@ -109,9 +110,15 @@ public voi

[GitHub] flink pull request #2546: [FLINK-4329] Fix Streaming File Source Timestamps/...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2546#discussion_r80901242 --- Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/AccumulatingAlignedProcessingTimeWindowOperatorTest.java

[GitHub] flink pull request #2546: [FLINK-4329] Fix Streaming File Source Timestamps/...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2546#discussion_r80902027 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/AsyncExceptionHandler.java --- @@ -18,12 +18,14 @@ package org

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529534#comment-15529534 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request #2546: [FLINK-4329] Fix Streaming File Source Timestamps/...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2546#discussion_r80904818 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileMonitoringTest.java --- @@ -224,7 +327,7 @@ public void testFilePathFilte

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529535#comment-15529535 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request #2546: [FLINK-4329] Fix Streaming File Source Timestamps/...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2546#discussion_r80901983 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/DefaultTimeServiceProvider.java --- @@ -99,7 +100,7 @@ public void

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529537#comment-15529537 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529539#comment-15529539 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529538#comment-15529538 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on a diff

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529536#comment-15529536 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on a diff

[GitHub] flink issue #2546: [FLINK-4329] Fix Streaming File Source Timestamps/Waterma...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2546 All in all some minor change requests, otherwise this seems good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529543#comment-15529543 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on the is

[GitHub] flink issue #2546: [FLINK-4329] Fix Streaming File Source Timestamps/Waterma...

2016-09-28 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2546 Actually, let me take a step back and understand a few things deeper, first. Who actually generates the watermarks (in ingestion time)? The operator that creates the file splits, or the operat

[jira] [Commented] (FLINK-4329) Fix Streaming File Source Timestamps/Watermarks Handling

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529556#comment-15529556 ] ASF GitHub Bot commented on FLINK-4329: --- Github user StephanEwen commented on the is

[jira] [Commented] (FLINK-4701) Unprotected access to cancelables in StreamTask

2016-09-28 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529571#comment-15529571 ] Stephan Ewen commented on FLINK-4701: - I don't understand this. The constructor does n

[jira] [Commented] (FLINK-4701) Unprotected access to cancelables in StreamTask

2016-09-28 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529577#comment-15529577 ] Ted Yu commented on FLINK-4701: --- How about other threads which may modify cancelables ? > U

[GitHub] flink issue #2559: [FLINK-4702] [kafka connector] Commit offets to Kafka asy...

2016-09-28 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2559 @StephanEwen On a second look, I think the `commitSpecificOffsetsToKafka` method was designed to commit synchronously in the first place. `AbstractFetcher` holds a Map of all current pending of

[jira] [Commented] (FLINK-4702) Kafka consumer must commit offsets asynchronously

2016-09-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529580#comment-15529580 ] ASF GitHub Bot commented on FLINK-4702: --- Github user tzulitai commented on the issue

  1   2   3   >