criptor.of().
But since - if I understand correctly - a single (topic,partition) can be split
into multiple workers, how can we make sure that we always commit offsets in
the correct order ?
Thanks a lot for your help,
Jean.
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtr
Hi John,Thanks for your
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtraders-corp/files/images/flow-traders-logo-mv.png]
Flow Traders B.V.
T: +31 20 799 6497
F: +31 20 799 6780
Jacob Bontiusplaats 9
1018 LL Amsterdam
Nederland
www.flowtraders.com<http://www.flowtraders.
committed.
I guess the reason for that is because in ReadFromKafkaDoFn in
theinitialRestriction(), the consumer has still a different name (prefixed with
"initialOffset").
Am I doing something wrong ?
Thanks,
Jean.
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtraders-
at happened? Using KafkaIO with ReadFromKafkaDoFn.java and
commitOffsetsFinalize should commit offsets of processed messages and if the
pipeline is restarted, should resume from the last committed offset. While
committing the offset wo...
Jean Wisser
Data Engineer
[https://www.flowtrader
s [100, 150), the
next restriction [150, 200) can be scheduled on a different worker ?
Thanks for your help.
Jean.
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtraders-corp/files/images/flow-traders-logo-mv.png]
Flow Traders B.V.
T: +31 20 799 6497
F: +31 20 799
ay of reading
kafka messages with beam ?
Thanks,
Jean Wisser.
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtraders-corp/files/images/flow-traders-logo-mv.png]
Flow Traders B.V.
T: +31 20 799 6497
F: +31 20 799 6780
Jacob Bontiusplaats 9
1018 LL Amsterdam
Nede
all files that were matched in the
matchAll() operation.
Any ideas on how to solve that ?
Thanks,
Jean Wisser.
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtraders-corp/files/images/flow-traders-logo-mv.png]
Flow Traders B.V.
T: +31 20 799 6497
F: +31 20 799 6780
Jaco
reshuffle step
right after parseFilesGenericRecords, but without any success.
Any ideas on how to resolve that ?
Thanks,
Jean Wisser.
Jean Wisser
Data Engineer
[https://www.flowtraders.com/sites/flowtraders-corp/files/images/flow-traders-logo-mv.png]
Flow Traders B.V.
T: +31 20 799 6497
