Re: possible data loss with Kafka I/O

2022-06-04 Thread Deepak Nagaraj
Hi Cham, Thank you for your response. One thing I didn't mention earlier is: all of this is with Beam's Flink runner. On Sat, Jun 4, 2022 at 9:55 AM Chamikara Jayalath wrote: > >> >> * Kafka consumer config: enable.auto.commit set to true (default), >> auto.offset.reset set to latest (default)

Re: possible data loss with Kafka I/O

2022-06-04 Thread Deepak Nagaraj
On Sat, Jun 4, 2022 at 3:35 PM Chamikara Jayalath wrote: > On Sat, Jun 4, 2022 at 1:55 PM Reuven Lax wrote: > >> Cham do you know if the Flunk runner uses the sdf version or the old >> version? >> > > I think that depends on whether the experiment "use_deprecated_read" was > specified or not. If