Conversely, we could consider making MM2 use "latest" in "legacy mode", and leave MM1 as it is? (Just thinking out loud.)
Ryanne On Mon, Apr 27, 2020 at 12:39 PM Jeff Widman <[email protected]> wrote: > Good questions: > > > *I agree that `auto.offset.reset="earliest"` would be a better default. > However, I am a little worried about backwardcompatibility. * > > Keep in mind that existing mirrormaker instances will *not* be affected for > topics they are currently consuming because they will already have saved > offsets. This will only affect mirrormakers that start consuming new > topics, for which they don't have a saved offset. In those cases, they will > stop seeing data loss when they first start consuming. My guess is the > majority of those new topics are going to be newly-created topics anyway, > so most of the time starting from the earliest simply prevents skipping the > first few seconds/minutes of data written to the topic. > > *What I am also wondering thought is, does this only affect MirrorMaker or > also MirrorMaker 2? * > > I checked and MM2 already sets `auto.offset.reset = 'earliest'` > < > https://github.com/apache/kafka/blob/d63eaaaa0181bb7b9b4f5ed088abc00d7b32aeb0/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java#L233 > > > . > > *Also, is it worth to change MirrorMaker now that **MirrorMaker 2 is > available?* > > Given that it's 1-line of code, doesn't affect existing instances, and > prevents data loss on new regex subscriptions, I think it's worth > setting... I basically view it as a bugfix rather than a feature change. > > I realize MM1 is deprecated, but there's still a lot of old mirrormakers > running, so flipping this now will ease the future transition to MM2 > because it brings the behavior of MM1 in line with MM2. > > Thoughts? > > > > On Sat, Apr 11, 2020 at 11:59 AM Matthias J. Sax <[email protected]> wrote: > > > Jeff, > > > > thanks for the KIP. I agree that `auto.offset.reset="earliest"` would be > > a better default. However, I am a little worried about backward > > compatibility. And even if the current default is not idea, users can > > still change it. > > > > What I am also wondering thought is, does this only affect MirrorMaker > > or also MirrorMaker 2? Also, is it worth to change MirrorMaker now that > > MirrorMaker 2 is available? > > > > > > -Matthias > > > > > > On 4/10/20 9:56 PM, Jeff Widman wrote: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-592%3A+MirrorMaker+should+replicate+topics+from+earliest > > > > > > It's a relatively minor change, only one line of code. :-D > > > > > > > > > > > > > > > -- > > *Jeff Widman* > jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) > <>< >
