I had to disable auto commit for it to work. I understand auto commit is
just for monitoring purpose so I assume it should be safe to run it like
that.
properties.put("enable.auto.commit", "false");
On Fri, Jan 31, 2020 at 1:09 PM RKandoji wrote:
> Hi,
>
> Thanks for the response.
> I'm actual
Hi,
Thanks for the response.
I'm actually using a different consumer group id for each consumer.
RK.
On Fri, Jan 31, 2020 at 12:20 PM Tzu-Li (Gordon) Tai
wrote:
> Hi,
>
> There are no upper limits on the number of Kafka consumers per job.
>
> For each one of your FlinkKafkaConsumers, are you u
Hi,
There are no upper limits on the number of Kafka consumers per job.
For each one of your FlinkKafkaConsumers, are you using the same group.id?
That could maybe explain why you are experiencing higher commit times as
you are adding more FlinkKafkaConsumers, as AFAIK on the broker side, the
com
Can someone please help me here.
Thanks
RK
On Thu, Jan 30, 2020 at 7:51 PM RKandoji wrote:
> Hi Team,
>
> I'm running into strange issue pasted below:
>
> Committing offsets to Kafka takes longer than the checkpoint interval.
> Skipping commit of previous offsets because newer complete checkpo
Hi Team,
I'm running into strange issue pasted below:
Committing offsets to Kafka takes longer than the checkpoint interval.
Skipping commit of previous offsets because newer complete checkpoint
offsets are available. This does not compromise Flink's checkpoint
integrity.
I read data from more
