Hi Ning, I mean by "when there are some messages left in the original cluster", when the consumer switches from the original cluster to the new cluster before entirely consuming the messages at the original cluster. This commit illustrates this situation. https://github.com/elakito/kafka/commit/9063db0f8c4a53f5d9764612af898981d499a7b7
regards, aki El mié, 13 ene 2021 a las 3:17, Ning Zhang (<ning2008w...@gmail.com>) escribió: > > Hello Aki, > > Can you elaborate on "when there are some messages left in the original > cluster" ? > > On 2021/01/12 13:01:40, Aki Yoshida <elak...@gmail.com> wrote: > > Hi Ryanne, > > Thanks for the information regarding the manual translation approach. > > But for 2.7.0, I have a question. I thought this translation would > > happen implicitly. > > I saw test testOneWayReplicationWithAutoOffsetSync of > > MirrorConnectorsIntegrationTest that is supposed to test this > > behavior. But I have an impression that the consumers can't be > > migrated when there are some messages left in the original cluster. I > > posted this question to users@kafka a while ago but I didn't receive > > any responses. > > https://lists.apache.org/thread.html/r0728e9ea89b2713865a33396f108027ac3f0949c2496583532ee963c%40%3Cusers.kafka.apache.org%3E > > Could you or someone comment on this? > > Thanks. > > Regards, aki > > > > El sáb, 2 ene 2021 a las 20:31, Ryanne Dolan (<ryannedo...@gmail.com>) > > escribió: > > > > > > Aki, that's right. Prior to 2.7, you can use the translateOffsets method > > > from within your client code, or you can write a little command-line tool > > > to do it. I've done the latter for Cloudera's srm-control tool, as > > > documented here: > > > https://docs.cloudera.com/csp/2.0.1/srm-using/topics/srm-migrating-consumer-groups.html > > > > > > Ryanne > > > > > > On Thu, Dec 17, 2020, 5:23 PM Aki Yoshida <elak...@gmail.com> wrote: > > > > > > > I'm answering to my own mail and would like to hear if this assumption > > > > is correct. > > > > it looks like I need 2.7.0 to have the automatic translation. > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-545%3A+support+automated+consumer+offset+sync+across+clusters+in+MM+2.0 > > > > for earlier versions, there is no way to use the console client but > > > > use java API RemoteClusterUtils.translateOffsets() to get the offset > > > > and re-consuming from the new topic. > > > > Is this correct? > > > > > > > > El jue, 17 dic 2020 a las 21:18, Aki Yoshida (<elak...@gmail.com>) > > > > escribió: > > > > > > > > > > Hi, > > > > > I have a question regarding how to migrate a consumer when the > > > > > subscribing topic has been migrated to the target Kafka broker. > > > > > Suppose a consumer is consuming from topic1 at the source Kafka > > > > > broker. When MM2 mirrors this topic using options > > > > > source.cluster.alias="", replication.policy.separator="", topic named > > > > > topic1 shows up at the target Kafka broker. > > > > > > > > > > When the consumer instance simply switches the bootstrap server to > > > > > start consuming from this topic1 at the target broker, the consumer > > > > > seems to start subscribing from the latest offset. I thought the > > > > > consumer offsets were also migrated to the target and the consumer > > > > > could simply start subscribing from the mirrored topic from the > > > > > continued offset. Did I miss some configuration parameters or does the > > > > > consumer need to perform some actions to be able to consume the > > > > > records seamlessly? > > > > > > > > > > I appreciate for your help. Thanks. > > > > > > > > > > regards, aki > > > > > >
