perf test on kafka and mirror2

Hi Kafka Experts I recently worked on kafka perf test, my kafka active/standby clusters and mirrormake2 are all built with sasl-ssl, kafka-producer-perf-test.sh setup with sasl-ssl too. I use kafka default settings, say, message.max.size, batch size...no changes... Test result shows active kaf

Python client failed to connect secured Kafka: SSL handshake failed: error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Hi Kafka Team Recently I moved Kafka cluster from CentOS8 to UbuntuServer20.04, same Kafka version(2.13-3.0.0), same Kafka configuration(check below), same JDK(openjdk-11-jdk) in server, but I get python client failed to connect. # SASL-SSL security.inter.broker.protocol=SASL_SSL sasl.enabled.me

回复: how to restore kafka and mm2 after disaster

even I successfully rebuild src kafka and topic/consumer group/acl in src kafka, when I test, I noticed consumer offset in src kafka is out of sync with dst kafka's, so how do we fail-back to src kafka if disaster is recovered, thanks 发件人: Calvin Chen

how to restore kafka and mm2 after disaster

Hi all I know kafka mm2 works good, it can mirror topic/message/offset to remote cluster, it is quite good, but I have question on how we restore cluster and mirror after disaster, let's say: I have src kafka mirror to dst kafka, and mm2 works well, then src kafka/zookeeper get disaster and co

Re: question about mm2 on consumer group offset mirroring

es consumer group offsets. > > Ryanne > >> On Thu, Sep 30, 2021, 9:18 AM Calvin Chen wrote: >> >> Hi all >> >> I have a question about the mirror make 2, on the consumer group offset >> mirroring, what is the duration for mm2 to detect consumer group offset &g

question about mm2 on consumer group

Hi all I see for mm2, mirrored topic name will have source kafka cluster put as prefix, but consumer group doesn't, why not put consumer group naming same as topic? is it because consumer group/offset mirroring not only from source-kafka to remote-kafka but also from remote-kafka to source-kafk

question about mm2 on consumer group offset mirroring

Hi all I have a question about the mirror make 2, on the consumer group offset mirroring, what is the duration for mm2 to detect consumer group offset change and mirror it to remote kafka consumer group? I have my mm2 code define as below: {{ kafka01_name }}->{{ kafka02_name }}.sync.group.off

kafka python client to create kafka user/acl info

Hi all I know there is confluent-kafka-python AdminClient to create/delete topic/config, can AdminClient also be used to create kafka user and acl info? could anybody share some link/example on this, thanks! -Calvin

Re: kafka mirror with TLS and SASL enabled

t unioned > together? Does the latest change win? etc. > > I agree this would be a nice feature tho. You might want to bring it up on > the dev list. > > Ryanne > >> On Thu, Jun 17, 2021, 10:24 AM Calvin Chen wrote: >> >> Hi all >> >> I have

kafka mirror with TLS and SASL enabled

Hi all I have a question, does kafka mirror2.0 mirror kafka users(created by kafka-configs.sh dynamically) and kafka acls(topic/group)? I setup below fields in mirror config file, and I think kafka mirror2.0 should mirror users and acls(topic/group) into remote cluster, but I see only part of

TSL and SASL for kafka and zookeeper

Hi all I have a question about enabling kafka and zookeeper on TSL and SASL together, that it will use TSL to do encryption and SASL to do auth, below are my config files(removed non-related info) zookeeper.conf: secureClientPort=2182 serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnx

Question about Kafka ACL

Hi all I have a question about kafka acl, is it possible to limit users to access a topic? For example, if person-a created kafka-user-a and grant kafka-user-a to access topic-a, and person-b knows there is topic-a, but he doesn't know the credential to access topic-a via kafka-user-a, so he c

kafka mm2 not replicate topic and data

Hi all I'm testing Kafka MM2, and noticed topic and data are not replicated, could anyone help, thanks a lot! My kafka version is kafka_2.13-2.8.0, two kafka cluster: sc2-dev-shared-kafka01 and sc2-dev-shared-kafka02, MM2 processes runs in each node of kafka clusters(3 MM2 processes in kafka01

Kafka SSL

Hi all I'm working on Kafka(kafka_2.13-2.7.0)cluster with SSL enabled, and I need help on Kafka broker config(I got error of connection failed) and client SSL config(I got error of SSL handshake failed). I setup Kafka and client SSL config by taking reference of Apache Kafka

Re: options for kafka cluster backup?

Hi all Thanks for the info, if you don’t mind, some more questions, thanks When enable mirror maker from primary to secondary, it will mirror all data, say topic message along with offset, so when primary Kafka is down, user can switch to secondary Kafka to continue where left in primary Kafka?

kafka log.retention.bytes

Hi all I have question about Kafka topic log retention on bytes, does the log.retention.bytes apply to each topic or it apply to all topics in broker? If it apply to each topic, then, when topic numbers keep growing in broker, how can we make sure total disk size for all topic logs will not exc

question about kafka topic

Hi, I have a question about kafka topic, recently we face kafka client sending message to kafka topic issue, got error about offset, and client can not send message to kafka cluster. My question is, since we configure kafka servers to one cluster, when cluster get message, will it try it best t