> Nicu
>
> From: Sönke Liebau [soenke.lie...@opencore.com.INVALID]
> Sent: Wednesday, February 21, 2018 4:59 PM
> To: users@kafka.apache.org
> Subject: Re: commiting consumed offsets synchronously (every message)
>
> Hi Nicu,
>
> committin
possibility of duplicate processing downstream of a topic.
Thanks for your help,
Nicu
From: Sönke Liebau [soenke.lie...@opencore.com.INVALID]
Sent: Wednesday, February 21, 2018 4:59 PM
To: users@kafka.apache.org
Subject: Re: commiting consumed offsets
Hi Nicu,
committing after every message and thus retrieving them with a batch size
of 1 will definitely make a huge difference in performance!
I've rigged a quick (and totally non academic) test which came up with the
following numbers:
Batching consumer - Consumed 1000490 records in 5 seconds
No
Hi,
In order to obtain an exactly-once semantics, we are thinking of doing
at-least-once processing, and then have a compensation mechanism to fix the
results in few minutes by correcting them by substracting the effects of the
duplicates. However, in order to do that, it seems that at least thi
