[jira] [Created] (FLINK-3036) Gelly's Graph.fromCsvReader method returns wrongly parameterized Graph

2015-11-17 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-3036: Summary: Gelly's Graph.fromCsvReader method returns wrongly parameterized Graph Key: FLINK-3036 URL: https://issues.apache.org/jira/browse/FLINK-3036 Project: Flink

[jira] [Created] (FLINK-3035) Redis as State Backend

2015-11-17 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-3035: -- Summary: Redis as State Backend Key: FLINK-3035 URL: https://issues.apache.org/jira/browse/FLINK-3035 Project: Flink Issue Type: New Feature Co

[jira] [Created] (FLINK-3034) Redis SInk Connector

2015-11-17 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-3034: -- Summary: Redis SInk Connector Key: FLINK-3034 URL: https://issues.apache.org/jira/browse/FLINK-3034 Project: Flink Issue Type: New Feature Comp

[jira] [Created] (FLINK-3033) Redis Source Connector

2015-11-17 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-3033: -- Summary: Redis Source Connector Key: FLINK-3033 URL: https://issues.apache.org/jira/browse/FLINK-3033 Project: Flink Issue Type: New Feature Co

[jira] [Created] (FLINK-3032) Flink does not start on Hadoop 2.7.1 (HDP), due to class conflict

2015-11-17 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3032: - Summary: Flink does not start on Hadoop 2.7.1 (HDP), due to class conflict Key: FLINK-3032 URL: https://issues.apache.org/jira/browse/FLINK-3032 Project: Flink

[jira] [Created] (FLINK-3031) Consistent Shutdown of Streaming Jobs

2015-11-17 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-3031: -- Summary: Consistent Shutdown of Streaming Jobs Key: FLINK-3031 URL: https://issues.apache.org/jira/browse/FLINK-3031 Project: Flink Issue Type: Bug

Re: [DISCUSSION] Consistent shutdown of streaming jobs

2015-11-17 Thread Stephan Ewen
Let do a separate JIRA, not overload this already tough pull request... On Fri, Nov 13, 2015 at 10:44 AM, Matthias J. Sax wrote: > I was thinking about this issue too and wanted to include it in my > current PR (I just about to rebase it to the current master... > https://github.com/apache/flink

[jira] [Created] (FLINK-3030) Enhance Dashboard to show Execution Attempts

2015-11-17 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3030: --- Summary: Enhance Dashboard to show Execution Attempts Key: FLINK-3030 URL: https://issues.apache.org/jira/browse/FLINK-3030 Project: Flink Issue Type: Improvem

[jira] [Created] (FLINK-3029) JobGraph is not connected correctly in webfrontend for streaming job

2015-11-17 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-3029: - Summary: JobGraph is not connected correctly in webfrontend for streaming job Key: FLINK-3029 URL: https://issues.apache.org/jira/browse/FLINK-3029 Project: Flink

[jira] [Created] (FLINK-3028) Cannot cancel restarting job via web frontend

2015-11-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3028: -- Summary: Cannot cancel restarting job via web frontend Key: FLINK-3028 URL: https://issues.apache.org/jira/browse/FLINK-3028 Project: Flink Issue Type: Improveme

[jira] [Created] (FLINK-3027) Cancel button in Flink job dashboard does not work

2015-11-17 Thread Krzysztof Zarzycki (JIRA)
Krzysztof Zarzycki created FLINK-3027: - Summary: Cancel button in Flink job dashboard does not work Key: FLINK-3027 URL: https://issues.apache.org/jira/browse/FLINK-3027 Project: Flink Is

[jira] [Created] (FLINK-3026) Publish the flink docker container to the docker registry

2015-11-17 Thread Omer Katz (JIRA)
Omer Katz created FLINK-3026: Summary: Publish the flink docker container to the docker registry Key: FLINK-3026 URL: https://issues.apache.org/jira/browse/FLINK-3026 Project: Flink Issue Type: T

Re: Redis connector for Flink

2015-11-17 Thread Matthias J. Sax
I was thinking about source, sink. But why not as a state backend, too. On 11/17/2015 12:08 PM, Stephan Ewen wrote: > Nice idea! > > Do you mean Redis as a state backend, a source, or a sink? > > On Tue, Nov 17, 2015 at 11:41 AM, Matthias J. Sax wrote: > >> Hi, >> >> I was just wondering if

Re: Redis connector for Flink

2015-11-17 Thread Stephan Ewen
Nice idea! Do you mean Redis as a state backend, a source, or a sink? On Tue, Nov 17, 2015 at 11:41 AM, Matthias J. Sax wrote: > Hi, > > I was just wondering if we should put a Redis connector for Flink on the > agenda. Or do we have a JIRA for this already? If not, just having a > JIRA might t

[jira] [Created] (FLINK-3025) Flink Kafka consumer may get stuck due to Kafka/Zookeeper client bug

2015-11-17 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-3025: - Summary: Flink Kafka consumer may get stuck due to Kafka/Zookeeper client bug Key: FLINK-3025 URL: https://issues.apache.org/jira/browse/FLINK-3025 Project: Flink

Redis connector for Flink

2015-11-17 Thread Matthias J. Sax
Hi, I was just wondering if we should put a Redis connector for Flink on the agenda. Or do we have a JIRA for this already? If not, just having a JIRA might trigger some external contribution for it :) -Matthias signature.asc Description: OpenPGP digital signature

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Stephan Ewen
sure On Tue, Nov 17, 2015 at 11:30 AM, Gyula Fóra wrote: > Should I open a JIRA for this? > > Gyula Fóra ezt írta (időpont: 2015. nov. 17., K, > 11:30): > > > Thanks for the quick response and thorough explanation :) > > > > Gyula > > > > Robert Metzger ezt írta (időpont: 2015. nov. 17., > > K

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Gyula Fóra
Thanks for the quick response and thorough explanation :) Gyula Robert Metzger ezt írta (időpont: 2015. nov. 17., K, 11:27): > I would try that approach first > > On Tue, Nov 17, 2015 at 11:26 AM, Stephan Ewen wrote: > > > Another idea: The bug is fixed in the Zookeeper Client that Kafka uses,

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Gyula Fóra
Should I open a JIRA for this? Gyula Fóra ezt írta (időpont: 2015. nov. 17., K, 11:30): > Thanks for the quick response and thorough explanation :) > > Gyula > > Robert Metzger ezt írta (időpont: 2015. nov. 17., > K, 11:27): > >> I would try that approach first >> >> On Tue, Nov 17, 2015 at 11:

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Robert Metzger
I would try that approach first On Tue, Nov 17, 2015 at 11:26 AM, Stephan Ewen wrote: > Another idea: The bug is fixed in the Zookeeper Client that Kafka uses, so > if we can bump the transitive dependency, that might fix it... > > On Tue, Nov 17, 2015 at 11:19 AM, Stephan Ewen wrote: > > > Hey

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Stephan Ewen
Another idea: The bug is fixed in the Zookeeper Client that Kafka uses, so if we can bump the transitive dependency, that might fix it... On Tue, Nov 17, 2015 at 11:19 AM, Stephan Ewen wrote: > Hey! > > The problem here is that there is no such thing as proper thread killing > in Java (at least

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Stephan Ewen
Hey! The problem here is that there is no such thing as proper thread killing in Java (at least it makes everything unstable if you do). Threads need to exit cooperatively. The Kafka Function calls simply are uninterruptibly stuck and never return (pretty bad bug in their Zookeeper Client). As fa

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Ufuk Celebi
I had a quick chat with Robert and Stephan. The problem is that the StreamTask cancellation needs to acquire a lock, which is held by the Kafka client in an infinite loop. At this point, I’m not sure what our options are here. Maybe Stephan or Robert can chime in… – Ufuk > On 17 Nov 2015, at

[jira] [Created] (FLINK-3024) TimestampExtractor Does not Work When returning Long.MIN_VALUE

2015-11-17 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-3024: --- Summary: TimestampExtractor Does not Work When returning Long.MIN_VALUE Key: FLINK-3024 URL: https://issues.apache.org/jira/browse/FLINK-3024 Project: Flink

Re: Failing kafka consumer unable to cancel

2015-11-17 Thread Ufuk Celebi
https://issues.apache.org/jira/browse/KAFKA-824 This has been fixed for Kafka’s 0.9.0 version. We should investigate why the job gets stuck though. Do you have a stack trace or any logs available? – Ufuk > On 17 Nov 2015, at 09:24, Gyula Fóra wrote: > > Hey guys, > > I ran into some issue w

[jira] [Created] (FLINK-3023) Show Flink version + commit id for -SNAPSHOT versions in web frontend

2015-11-17 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3023: - Summary: Show Flink version + commit id for -SNAPSHOT versions in web frontend Key: FLINK-3023 URL: https://issues.apache.org/jira/browse/FLINK-3023 Project: Flink

[jira] [Created] (FLINK-3022) Broken link 'Working With State' in Fault Tolerance Section of Stream Programming Guide

2015-11-17 Thread Suneel Marthi (JIRA)
Suneel Marthi created FLINK-3022: Summary: Broken link 'Working With State' in Fault Tolerance Section of Stream Programming Guide Key: FLINK-3022 URL: https://issues.apache.org/jira/browse/FLINK-3022

[jira] [Created] (FLINK-3021) Job submission times out due to classloading issue on JobManager

2015-11-17 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3021: - Summary: Job submission times out due to classloading issue on JobManager Key: FLINK-3021 URL: https://issues.apache.org/jira/browse/FLINK-3021 Project: Flink

Failing kafka consumer unable to cancel

2015-11-17 Thread Gyula Fóra
Hey guys, I ran into some issue with the kafka consumers. I am reading from more than 50 topics with parallelism 1, and while running the job I got the following exception during the checkpoint notification (offset committing): java.lang.RuntimeException: Error while confirming checkpoint at org