I am not very sure about the isolation.level setting. However duplicates may cause due to the commit failed on the consumer side. Please do read about max.poll.interval.ms and max.poll.records settings. You may get some valuable inputs.
Recently i solved duplicates issue in my consumer by tuning above values. Hope it helps. Regards, Sunil. On Thu, 29 Oct 2020 at 3:38 PM, usha rani <ushanaraya...@gmail.com> wrote: > Hi Team, > > Recently we ended up having huge duplicates sending out of the connector > due to the segment roll out ended in creating out of range issue. To avoid > the occurance of the above issue we made couple of changes as part of it we > are planning to change 'isolation.level' to 'read_committed'. > > So before making that change I wanted to understand why the default of > 'isolation.level' is 'read_uncommitted' and not 'read_committed'. When I > see read_uncommitted end up in dirty read and producing duplicates. > > Thanks, > Usha >
