Re: EFO KinesisIO watermarking doubt

2023-10-01 Thread Sachin Mittal
Hi, I have filed an issue: https://github.com/apache/beam/issues/28760 I have also created a PR (based of our local fix for this): https://github.com/apache/beam/pull/28763 This can serve as a start. Thanks Sachin On Mon, Oct 2, 2023 at 2:54 AM Pavel Solomin wrote: > Hello, sorry for the late

Re: EFO KinesisIO watermarking doubt

2023-10-01 Thread Pavel Solomin
Hello, sorry for the late reply. EFOKinesisReader implemented the same logic of timestamps non-EFO KinesisReader had. At the time of EFO implementation more careful evaluation of the records' timestamps was out of context. Can you please create an issue at https://github.com/apache/beam/issues ?

EFO KinesisIO watermarking doubt

2023-07-20 Thread Sachin Mittal
Hi, We are implementing EFO Kinesis IO reader provided by apache beam. I see that in code that for implementation of getCurrentTimestamp we always return getApproximateArrivalTimestamp and not the event time which we may have set for that record using withCustomWatermarkPolicy. Please refer: https