Re: Kinesis Connector Dependency Problems

2016-11-08 Thread Fabian Hueske
t; .operators.StreamSource.run(StreamSource.java:80) > >>> > >>> at org.apache.flink.streaming.api > >>> .operators.StreamSource.run(StreamSource.java:53) > >>> > >>>

Re: Kinesis Connector Dependency Problems

2016-11-08 Thread Foster, Craig
mSource.java:80) >>> >>> at org.apache.flink.streaming.api >>> .operators.StreamSource.run(StreamSource.java:53) >>> >>> at org.apache.flink.streaming.run >>> time.tasks.SourceStreamTask.run(SourceStreamTask.java:56) >>&

Re: Kinesis Connector Dependency Problems

2016-11-08 Thread Steffen Hausmann
>>> .operators.StreamSource.run(StreamSource.java:80) >>> >>> at org.apache.flink.streaming.api >>> .operators.StreamSource.run(StreamSource.java:53) >>> >>> at org.apache.flink.streaming.run >>> time.tasks.SourceS

Re: Kinesis Connector Dependency Problems

2016-11-08 Thread Fabian Hueske
at org.apache.flink.streaming.run >> time.tasks.SourceStreamTask.run(SourceStreamTask.java:56) >> >> at org.apache.flink.streaming.run >> time.tasks.StreamTask.invoke(StreamTask.java:266) >> >> at org.apache.flink.runti

Re: Kinesis Connector Dependency Problems

2016-11-08 Thread Till Rohrmann
at org.apache.flink.runtime.taskmanager.Task.run(Task. > java:585) > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > *From: *Robert Metzger > *Reply-To: *"user@flink.apache.org" > *Date: *Friday, November 4,

Re: Kinesis Connector Dependency Problems

2016-11-04 Thread Robert Metzger
Thank you for helping to investigate the issue. I've filed an issue in our bugtracker: https://issues.apache.org/jira/browse/FLINK-5013 On Wed, Nov 2, 2016 at 10:09 PM, Justin Yan wrote: > Sorry it took me a little while, but I'm happy to report back that it > seems to be working properly with E

Re: Kinesis Connector Dependency Problems

2016-11-02 Thread Justin Yan
Sorry it took me a little while, but I'm happy to report back that it seems to be working properly with EMR 4.8. It seems so obvious in retrospect... thanks again for the assistance! Cheers, Justin On Tue, Nov 1, 2016 at 11:44 AM, Robert Metzger wrote: > Hi Justin, > > thank you for sharing t

Re: Kinesis Connector Dependency Problems

2016-11-01 Thread Robert Metzger
Hi Justin, thank you for sharing the classpath of the Flink container with us. It contains what Till was already expecting: An older version of the AWS SDK. If you have some spare time, could you quickly try to run your program with a newer EMR version, just to validate our suspicion? If the erro

Re: Kinesis Connector Dependency Problems

2016-11-01 Thread Justin Yan
Hi there, We're using EMR 4.4.0 -> I suppose this is a bit old, and I can migrate forward if you think that would be best. I've appended the classpath that the Flink cluster was started with at the end of this email (with a slight improvement to the formatting to make it readable). Willing to po

Re: Kinesis Connector Dependency Problems

2016-11-01 Thread Till Rohrmann
Hi Justin, I think this might be a problem in Flink's Kinesis consumer. The Flink Kinesis consumer uses the aws-java-sdk version 1.10.71 which indeed contains the afore mentioned methods. However, already version 1.10.46 no longer contains this method. Thus, I suspect, that Yarn puts some older ve

Kinesis Connector Dependency Problems

2016-10-31 Thread Justin Yan
Hi all - first time on the mailing list, so my apologies if I break protocol on anything. Really excited to be using Flink, and hoping to be active here in the future! Also, apologies for the length of this email - I tried to include details but may have gone overboard. The gist of my problem is