With the discussion about a 3.0 release and deprecating the old MM, I am wondering if it's worth to do anything.
Please should just switch to MM2 that has a better default. Thoughts? On 4/27/20 10:48 AM, Ryanne Dolan wrote: > Conversely, we could consider making MM2 use "latest" in "legacy mode", and > leave MM1 as it is? (Just thinking out loud.) > > Ryanne > > On Mon, Apr 27, 2020 at 12:39 PM Jeff Widman <[email protected]> wrote: > >> Good questions: >> >> >> *I agree that `auto.offset.reset="earliest"` would be a better default. >> However, I am a little worried about backwardcompatibility. * >> >> Keep in mind that existing mirrormaker instances will *not* be affected for >> topics they are currently consuming because they will already have saved >> offsets. This will only affect mirrormakers that start consuming new >> topics, for which they don't have a saved offset. In those cases, they will >> stop seeing data loss when they first start consuming. My guess is the >> majority of those new topics are going to be newly-created topics anyway, >> so most of the time starting from the earliest simply prevents skipping the >> first few seconds/minutes of data written to the topic. >> >> *What I am also wondering thought is, does this only affect MirrorMaker or >> also MirrorMaker 2? * >> >> I checked and MM2 already sets `auto.offset.reset = 'earliest'` >> < >> https://github.com/apache/kafka/blob/d63eaaaa0181bb7b9b4f5ed088abc00d7b32aeb0/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java#L233 >>> >> . >> >> *Also, is it worth to change MirrorMaker now that **MirrorMaker 2 is >> available?* >> >> Given that it's 1-line of code, doesn't affect existing instances, and >> prevents data loss on new regex subscriptions, I think it's worth >> setting... I basically view it as a bugfix rather than a feature change. >> >> I realize MM1 is deprecated, but there's still a lot of old mirrormakers >> running, so flipping this now will ease the future transition to MM2 >> because it brings the behavior of MM1 in line with MM2. >> >> Thoughts? >> >> >> >> On Sat, Apr 11, 2020 at 11:59 AM Matthias J. Sax <[email protected]> wrote: >> >>> Jeff, >>> >>> thanks for the KIP. I agree that `auto.offset.reset="earliest"` would be >>> a better default. However, I am a little worried about backward >>> compatibility. And even if the current default is not idea, users can >>> still change it. >>> >>> What I am also wondering thought is, does this only affect MirrorMaker >>> or also MirrorMaker 2? Also, is it worth to change MirrorMaker now that >>> MirrorMaker 2 is available? >>> >>> >>> -Matthias >>> >>> >>> On 4/10/20 9:56 PM, Jeff Widman wrote: >>>> >>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-592%3A+MirrorMaker+should+replicate+topics+from+earliest >>>> >>>> It's a relatively minor change, only one line of code. :-D >>>> >>>> >>>> >>> >>> >> >> -- >> >> *Jeff Widman* >> jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) >> <>< >> >
signature.asc
Description: OpenPGP digital signature
