This is somewhat implied in
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/datastream/kafka/#consumer-offset-committing.
/> Note that Kafka source does //*NOT*//rely on committed offsets for
fault tolerance. Committing offset is only for exposing the progress of
con
I would suggest updating the documentation to include that statement.
I imagine dynamic partition discovery has no effect on this?
Regards,
Alexis.
Am Do., 21. Juli 2022 um 10:03 Uhr schrieb Chesnay Schepler <
ches...@apache.org>:
> Flink only reads the offsets from Kafka when the job is initia
Flink only reads the offsets from Kafka when the job is initially
started from a clear slate.
Once checkpoints are involved it only relies on offsets stored in the state.
On 20/07/2022 14:51, Alexis Sarda-Espinosa wrote:
Hello again,
I just performed a test
using OffsetsInitializer.committedO
Hello again,
I just performed a test
using OffsetsInitializer.committedOffsets(OffsetResetStrategy.LATEST). I
did a few tests in the following order, and I noticed a few weird things.
Note that our job uses Processing Time windows, so watermarks are
irrelevant.
1. After the job had been running f
Hi Yaroslav,
The test I did was just using earliest, I'll test with committed offset
again, thanks.
Regards,
Alexis.
On Thu, 14 Jul 2022, 20:49 Yaroslav Tkachenko, wrote:
> Hi Alexis,
>
> Do you use OffsetsInitializer.committedOffsets() to specify your Kafka
> consumer offsets? In this case, i
Hi Alexis,
Do you use OffsetsInitializer.committedOffsets() to specify your Kafka
consumer offsets? In this case, it should get the offsets from Kafka and
not the state.
On Thu, Jul 14, 2022 at 11:18 AM Alexis Sarda-Espinosa <
sarda.espin...@gmail.com> wrote:
> Hello,
>
> Regarding the new Kafka
