Hi An, Try setting consumer.auto.offset.reset=earliest This will mean if the consumers in the MM2 tasks don’t have an offset for a topic partition they will start at the beginning. If you’ve already enabled the below you might have to use a new group.id so that the consumers no longer have an offset that is valid. Since when you stated MM2 with below config it will now have an offset stored in the offset storage topic for each topic partition being consumed and therefore the auto.offset.reset config won’t be applied. Thanks, Jamie
Sent from the all-new AOL app for iOS On Tuesday, June 14, 2022, 12:19 am, An, Hongguo (CORP) <hongguo...@adp.com.INVALID> wrote: Hi: I am trying to use MM2 ( 3.3.0) to migration Kafka to a new server, but looks like it only copys new data, any existing data is not copied. Here is the connection properties: clusters=CDL,PC CDL.bootstrap.servers=xxx PC.bootstrap.servers=xxx PC.security.protocol=SSL CDL->PC.enabled=true topics= .* groups= .* topic.blacklist= .*[\~\.]internal, .*\.replica, __.*, heartbeats,nas\.vantage.*,INTEGRATION.*, ADPRM\..*, ADP_LIFION_INTEGRATION\..*, AES\..*, BENEFITS\..*, DMX\..*, NASBILLING\..*, TAAS\..* groups.blacklist= console-consumer-.*, connect-.*, __.* tasks.max= 10 #emit.checkpoints.enabled= true #sync.group.offsets.enabled= true #offset.lag.max= 9223372036854775807 replication.factor= 3 checkpoints.topic.replication.factor= 3 heartbeats.topic.replication.factor= 3 offset-syncs.topic.replication.factor= 3 offset.storage.replication.factor= 3 status.storage.replication.factor= 3 config.storage.replication.factor= 3 transaction.state.log.replication.factor= 3 Thanks in advance for any help This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, notify the sender immediately by return email and delete the message and any attachments from your system.
