[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224651#comment-17224651
]
Sarita edited comment on KAFKA-7500 at 11/5/20, 6:24 AM:
---------------------------------------------------------
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
{quote}quote
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets-test
config.storage.topic=connect-configs-test
status.storage.topic=connect-status-test
offset.flush.interval.ms=300000
producer.buffer.memory=1234
producer.ssl.truststore.password=truststore_password
producer.ssl.truststore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
producer.ssl.keystore.password=keystore_password
producer.ssl.keystore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
quote
{quote}
When i run below command, I can see worker connector created:
{quote}{{}}}}
\{{ ```nohup sh connect-distributed
../etc/kafka/connect-distributed-1.properties &``` }}
{{
{quote}
When I try to create a connector using POST call, I start seeing message as
{quote}{{}}}}
```WARN [Producer clientId=connector-producer-MM9-0] Bootstrap broker
abc-broker:9093 (id: -1 rack: null) disconnected
(org.apache.kafka.clients.NetworkClient:1037)```
{{
{quote}
json content for POST call is as below
{quote}{{ "name": "MM9",
"config": { "connector.class":
"org.apache.kafka.connect.mirror.MirrorSourceConnector",
"tasks.max": 3,
"topics": "messaging_ops_mm8",
"errors.log.enable": true,
"errors.log.include.messages": true,
"key.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"clusters": "peach, tails",
"source.cluster.alias": "peach",
"target.cluster.alias": "tails",
"source.cluster.bootstrap.servers": "xyz-broker:9093",
"source.cluster.producer.bootstrap.servers": "xyz-broker:9093",
"sync.topic.acls.enabled": "false",
"emit.checkpoints.interval.seconds": "1",
"groups": "consumer-group-.*",
"source.cluster.admin.bootstrap.servers": "xyz-broker:9093",
"source.cluster.consumer.bootstrap.servers": "xyz-broker:9093",
"source.cluster.security.protocol": "SASL_SSL",
"source.cluster.sasl.mechanism": "PLAIN",
"source.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser-password';",
"source.cluster.ssl.truststore.password" : "truststorepassword",
"source.cluster.ssl.truststore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"source.cluster.ssl.keystore.password" : "keystorepassword",
"source.cluster.ssl.keystore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"target.cluster.bootstrap.servers": "abc-broker:9093",
"target.cluster.producer.bootstrap.servers": "abc-broker:9093",
"enabled": "true",
"target.cluster.admin.bootstrap.servers": "abc-broker:9093",
"target.cluster.consumer.bootstrap.servers": "abc-broker:9093",
"name": "MirrorSourceConnector",
"emit.heartbeats.interval.seconds": "1",
"target.cluster.security.protocol": "SASL_SSL",
"target.cluster.sasl.mechanism": "PLAIN",
"target.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser-password';",
"target.cluster.ssl.truststore.password" : "truststorepassword",
"target.cluster.ssl.truststore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"target.cluster.ssl.keystore.password" : "keystore-password",
"target.cluster.ssl.keystore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks" }}
{quote}
Note the bootstrap-server added in connect-distributed.properties file and the
target cluster bootstrap-server are same(abc-broker).
I have added all the SASL credentials, did a swap of source and destination
cluster in the json but I continue to get this broker disconnected WARNING.
We are on kafka version 0.11.0.3.
What is it that we are missing?
was (Author: saritago):
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
{quote}quote
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets-test
config.storage.topic=connect-configs-test
status.storage.topic=connect-status-test
offset.flush.interval.ms=300000
producer.buffer.memory=1234
producer.ssl.truststore.password=truststore_password
producer.ssl.truststore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
producer.ssl.keystore.password=keystore_password
producer.ssl.keystore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
quote
{quote}
When i run below command, I can see worker connector created:
{quote}{{}}}}
\{{ ```nohup sh connect-distributed
../etc/kafka/connect-distributed-1.properties &``` }}
{{
{quote}
When I try to create a connector using POST call, I start seeing message as
{quote}{{}}}}
```WARN [Producer clientId=connector-producer-MM9-0] Bootstrap broker
abc-broker:9093 (id: -1 rack: null) disconnected
(org.apache.kafka.clients.NetworkClient:1037)```
{{
{quote}
json content for POST call is as below
{quote}{{}}}}
\{{ ```}}
{{ {{ "name": "MM9",}}
\{{ "config":}}
Unknown macro: \{ "connector.class"}
}
```
{{
{quote}
Note the bootstrap-server added in connect-distributed.properties file and the
target cluster bootstrap-server are same(abc-broker).
I have added all the SASL credentials, did a swap of source and destination
cluster in the json but I continue to get this broker disconnected WARNING.
We are on kafka version 0.11.0.3.
What is it that we are missing?
> MirrorMaker 2.0 (KIP-382)
> -------------------------
>
> Key: KAFKA-7500
> URL: https://issues.apache.org/jira/browse/KAFKA-7500
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect, mirrormaker
> Affects Versions: 2.4.0
> Reporter: Ryanne Dolan
> Assignee: Ryanne Dolan
> Priority: Major
> Labels: pull-request-available, ready-to-commit
> Fix For: 2.4.0
>
> Attachments: Active-Active XDCR setup.png
>
>
> Implement a drop-in replacement for MirrorMaker leveraging the Connect
> framework.
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0]
> [https://github.com/apache/kafka/pull/6295]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
