Re: timeout exception when consuming from kafka

2019-07-27 Thread Yitzchak Lieberman
Hi. Turned out that the cause was non-replicated (replication factor = 1) topics in Kafka. On Wed, Jul 24, 2019 at 4:20 PM Yitzchak Lieberman < yitzch...@sentinelone.com> wrote: > Hi. > > Do we have an idea for this exception? > > Thanks, > Yitzchak. > > On Tue, Jul 23, 2019 at 12:59 PM Fabian H

Re: timeout exception when consuming from kafka

2019-07-24 Thread Yitzchak Lieberman
Hi. Do we have an idea for this exception? Thanks, Yitzchak. On Tue, Jul 23, 2019 at 12:59 PM Fabian Hueske wrote: > Hi Yitzchak, > > Thanks for reaching out. > I'm not an expert on the Kafka consumer, but I think the number of > partitions and the number of source tasks might be interesting t

Re: timeout exception when consuming from kafka

2019-07-23 Thread Fabian Hueske
Hi Yitzchak, Thanks for reaching out. I'm not an expert on the Kafka consumer, but I think the number of partitions and the number of source tasks might be interesting to know. Maybe Gordon (in CC) has an idea of what's going wrong here. Best, Fabian Am Di., 23. Juli 2019 um 08:50 Uhr schrieb Y

Re: timeout exception when consuming from kafka

2019-07-22 Thread Yitzchak Lieberman
Hi. Another question - what will happen during a triggered checkpoint if one of the kafka brokers is unavailable? Will appreciate your insights. Thanks. On Mon, Jul 22, 2019 at 12:42 PM Yitzchak Lieberman < yitzch...@sentinelone.com> wrote: > Hi. > > I'm running a Flink application (version 1.8

timeout exception when consuming from kafka

2019-07-22 Thread Yitzchak Lieberman
Hi. I'm running a Flink application (version 1.8.0) that uses FlinkKafkaConsumer to fetch topic data and perform transformation on the data, with state backend as below: StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); env.enableCheckpointing(5_000, Ch