Re: Data loss in Flink Kafka Pipeline

keys. There are multiple GroupByKey transformations in pipeline. After reading BEAM-3225, I assume that this is the bug in the runner. Thanks for connecting with Aljoscha. :) Hi Aljoscha, I will share the code with you in another mail thread. Thanks & regards, Nishu On Fri, Dec 8, 2017 at

Re: Data loss in Flink Kafka Pipeline

trigger fires. So I doubt if triggers are causing the issue. Thanks & regards, Nishu On Thu, Dec 7, 2017 at 11:47 PM, Fabian Hueske wrote: > Hi Nishu, > > the data loss might be caused by the fact that processing time triggers do > not fire when the program terminates. > So,

Re: Data loss in Flink Kafka Pipeline

, Nishu On Thu, Dec 7, 2017 at 7:13 PM, Chen Qin wrote: > Nishu > > You might consider sideouput with metrics at least after window. I would > suggest having that to catch data screw or partition screw in all flink > jobs and amend if needed. > > Chen > > On Thu, De

Data loss in Flink Kafka Pipeline

hanks in advance! -- Thanks & Regards, Nishu Tayal

Error while setting up flink on HDInsight cluster

ks in advance. -- Thanks & Regards, Nishu Tayal

Flink session on yarn

flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-session-on-Yarn-ClassNotFoundException-td15222.html I am using HDInsight Cluster(HDP3.6). According to the documentation, set HADOOP_CONF_DIR and YARN_CONF_DIR as well. Any inputs will be really helpful. Thanks! -- Thanks & Regards, Nishu Tayal