Question about MM2's source-cluster.checkpoints.internal topic

Hi, I am using MM2 version 3.1.0 for a uni-directional mirroring of source-cluster -> target-cluster. I am confused with the values found in the source-cluster.checkpoints.internal topic at the target cluster when the source's earliest offset is not 0. In my test setup, I prepared a topic samples.

Re: When using MM2, how should the consumer be switched from source to target?

fka/commit/9063db0f8c4a53f5d9764612af898981d499a7b7 regards, aki El mié, 13 ene 2021 a las 3:17, Ning Zhang () escribió: > > Hello Aki, > > Can you elaborate on "when there are some messages left in the original > cluster" ? > > On 2021/01/12 13:01:40, Aki Yoshida

Re: When using MM2, how should the consumer be switched from source to target?

little command-line tool > to do it. I've done the latter for Cloudera's srm-control tool, as > documented here: > https://docs.cloudera.com/csp/2.0.1/srm-using/topics/srm-migrating-consumer-groups.html > > Ryanne > > On Thu, Dec 17, 2020, 5:23 PM Aki Yoshida wrote: &g

Re: --override option for bin/connect-distributed.sh

ay/KAFKA/Kafka+Improvement+Proposals > > On Mon, Dec 21, 2020 at 8:57 PM Aki Yoshida wrote: > > > Hi Kafka team, > > I think the --override option of Kafka is very practical in starting > > Kafka for various situations without changing the properties file. I > > missed

--override option for bin/connect-distributed.sh

Hi Kafka team, I think the --override option of Kafka is very practical in starting Kafka for various situations without changing the properties file. I missed this feature in Kafka-Connect and I wanted to have it, so I created a patch in this commit in my forked repo. https://github.com/elakito/ka

MirrorConnectorsIntegrationTest's testOneWayReplicationWithAutoOffsetSync

Hi, I am new to MirrorMaker and I have been trying to understand its behavior, in particular about its consumer offset synchronization. I saw KIP-545 https://cwiki.apache.org/confluence/display/KAFKA/KIP-545%3A+support+automated+consumer+offset+sync+across+clusters+in+MM+2.0, which talks about th

Re: When using MM2, how should the consumer be switched from source to target?

there is no way to use the console client but use java API RemoteClusterUtils.translateOffsets() to get the offset and re-consuming from the new topic. Is this correct? El jue, 17 dic 2020 a las 21:18, Aki Yoshida () escribió: > > Hi, > I have a question regarding how to migrate a con

When using MM2, how should the consumer be switched from source to target?

Hi, I have a question regarding how to migrate a consumer when the subscribing topic has been migrated to the target Kafka broker. Suppose a consumer is consuming from topic1 at the source Kafka broker. When MM2 mirrors this topic using options source.cluster.alias="", replication.policy.separator=

Using FileConfigProvider with restricted path?

Hi, I am wondering if there is a possibility to restrict the path used for the FileConfigProvider's value in the configuration e.g., ${file:/xxxpath/xxxfile.properties:xxxname}. I am looking for a way to restrict this path. Thanks regards, aki

Re: swagger/openapi spec for Kafka-Connect REST API?

API? > > Hi, > > I'm also searching for this. If you find it, please let me know. > Thank you. > -- > Miguel Silvestre > > > On Mon, May 18, 2020 at 8:59 AM Aki Yoshida wrote: > > > I am looking for the openapi spec for Kafka-Connect API but I couldn'

swagger/openapi spec for Kafka-Connect REST API?

I am looking for the openapi spec for Kafka-Connect API but I couldn't find one. If it is available somewhere, could someone tell me where I can find it? regards, aki

Re: question about property advertised.listeners

=PLAINTEXT://:10092 > > Since you mentioned containers, are you using the Confluent's docker > containers? If yes, the above configs should be taken care of (via an > environment variable). > > Hope this helps. > > On Tue, Nov 29, 2016 at 9:56 AM, Aki Y

Looking for some info on kafka homepage

Hi, I am looking for the following information on Kafka's project home page but am not able to find them. The home page seems to have been updated recently (looks really nice ;-) and maybe I just don't know where to navigate through. The information that I would like to find are: - List of Kafka r

question about property advertised.listeners

I have a question regarding how to use advertised.listeners to expose a different port to the outside of the container where the broker is running. When using advertised.listeners to register a different port as the actual broker port, do we need a port forwarding from this configured port to the

ZK path name mentioned in the online Replication Design document seems to be incorrect

I am trying to understand the replication procedure and saw this document https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3 describing: LeaderAndISR path: stores leader and ISR of a partition /brokers/topics/[topic]/[partition_id]/leaderAndISR --> {leader_epoc