Thanks Gordon! This indeed seems like the cause of the issue. I've ran the program using 1.5.0, after building the appropriate connector, and it's working as expected.
Wondering how difficult is it to upgrade the 1.4 connector to a newer KPL version, as this kind of blocks running on EMR and producing to Kinesis... :-) Alexey On Thu, Jun 14, 2018 at 12:20 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > Hi, > > This could be related: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-4-and-below-STOPS-writing-to-Kinesis-after-June-12th-td22687.html#a22701 > . > > Shortly put, the KPL library version used by default in the 1.4.x Kinesis > connector, is no longer supported by AWS. > Users would need to use a upgraded version >= 0.12.6 and build the Kinesis > connector for the producer to work. > > We should probably add a warning about this in the Kinesis connector docs. > > Cheers, > Gordon > > On 14 June 2018 at 9:16:04 PM, Lasse Nedergaard (lassenederga...@gmail.com) > wrote: > > Hi. > > We see the same error and to my understanding it’s a known error from > Amazon. See. > https://github.com/awslabs/amazon-kinesis-producer/issues/39#issuecomment-396219522 > > We don’t have a workaround and haven’t found the reason for the exception. > It is one off the reason why we move to Kafka in the near future. > > Med venlig hilsen / Best regards > Lasse Nedergaard > > > Den 14. jun. 2018 kl. 20.24 skrev Alexey Tsitkin <alexey.tsit...@gmail.com > >: > > Hi, > I'm trying to run a simple program which consumes from one kinesis stream, > does a simple transformation, and produces to another stream. > Running on Flink 1.4.0. > > Code can be seen here (if needed I can also paste it directly on this > thread): > > https://stackoverflow.com/questions/50847164/flink-producing-to-kinesis-not-working > > Consuming the source stream works great, but trying to use the producer > causes the exception: > > *org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.DaemonException: > The child process has been shutdown and can no longer accept messages.* > * at > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.add(Daemon.java:176)* > * at > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.addUserRecord(KinesisProducer.java:477)* > * at > org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer.invoke(FlinkKinesisProducer.java:248)* > * ...* > > > Did anyone have something similar? > Or is there any way to debug the daemon itself, to understand the source > of the error? > > As you can see, this is a trivial example, which I mostly copy-pasted from > the documentation. > > Thanks, > Alexey > >
