Re: Problem producing to Kinesis

2018-06-15 Thread Sandybayev, Turar (CAI - Atlanta)
apache.org" Subject: Re: Problem producing to Kinesis @Alexey If you’d like to stick to 1.4.x for now, you can just do: `mvn clean install -Daws.kinesis-kpl-version=0.12.6` when building the Kinesis connector, to upgrade the KPL version used. I think we should add this to the documentatio

Re: Problem producing to Kinesis

2018-06-15 Thread Tzu-Li (Gordon) Tai
@Alexey If you’d like to stick to 1.4.x for now, you can just do: `mvn clean install -Daws.kinesis-kpl-version=0.12.6` when building the Kinesis connector, to upgrade the KPL version used. I think we should add this to the documentation. Here’s a JIRA to track that -  https://issues.apache.org/j

Re: Problem producing to Kinesis

2018-06-15 Thread dyana . rose
Porting and rebuilding 1.4.x isn't a big issue. I've done it on our fork, back when I reported the upcoming issue and we're running fine. https://github.com/SaleCycle/flink/commit/d943a172ae7e6618309b45df848d3b9432e062d4 Ignore the circleci file of course, and the rest are the changes that I bac

Re: Problem producing to Kinesis

2018-06-14 Thread Alexey Tsitkin
Thanks Gordon! This indeed seems like the cause of the issue. I've ran the program using 1.5.0, after building the appropriate connector, and it's working as expected. Wondering how difficult is it to upgrade the 1.4 connector to a newer KPL version, as this kind of blocks running on EMR and prod

Re: Problem producing to Kinesis

2018-06-14 Thread Tzu-Li (Gordon) Tai
Hi, This could be related:  http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-4-and-below-STOPS-writing-to-Kinesis-after-June-12th-td22687.html#a22701. Shortly put, the KPL library version used by default in the 1.4.x Kinesis connector, is no longer supported by AWS.

Re: Problem producing to Kinesis

2018-06-14 Thread Lasse Nedergaard
Hi. We see the same error and to my understanding it’s a known error from Amazon. See. https://github.com/awslabs/amazon-kinesis-producer/issues/39#issuecomment-396219522 We don’t have a workaround and haven’t found the reason for the exception. It is one off the reason why we move to Kafka i