Hi Team,
Today, In our production cluster, we faced an issue with Kafka (Old offsets was
getting pulled from spark streaming application) and couldn't debug the issue
using kafka_consumer_group.sh CLI.
Whenever we execute the below command to list the consumer groups, it is
working fine. However, whenever we try to describe the consumer group to get to
know the offset details, it didn't work (Nothing is getting displayed. Just
blank).
Command to list the consumer group (Working):
/opt/kafka/kafka_2.11-2.0.0/bin/kafka-consumer-groups.sh \
--bootstrap-server broker1:2345,broker2:2345,broker3:2345 \
--list \
--command-config /opt/kafka/kafka_2.11-2.0.0/config/jaas_config.conf
Command to list the consumer group (Not Working):
/opt/kafka/kafka_2.11-2.0.0/bin/kafka-consumer-groups.sh \
--bootstrap-server broker1:2345,broker2:2345,broker3:2345 \
--describe \
--group
spark-kafka-source-f8e218d5-16d2-4e63-a25c-2f96fabb2809-605351645-driver-0 \
--command-config /opt/kafka/kafka_2.11-2.0.0/config/jaas_config.conf
Jass Config File:
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule
required \
username="admin" \
password="123@admin";
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
exclude.internal.topics=false
Kindly help us to monitor our Kafka cluster in case of any issues.
Details:
Kafka Version: 2.0.0
Security:
sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
security.inter.broker.protocol=SASL_PLAINTEXT
Please let us know in case of any other details required from our end.
Regards,
AnnPricksEdmund
Ann Pricks
DBA Admin
BIG DATA
[https://www.dt.ae/signature/divider.jpg]
[Map] DT Building 8th Street, Al Khabaisi, 4286, Dubai, AE
[Phone] Ext: 152
[Phone] +971 4 222 4222
[Email] pri...@dt.ae
[Web] www.dt.ae
[https://www.dt.ae/signature/free-logo%20copy.png]
[https://www.facebook.com/DubaiTechnologies/]<https://www.facebook.com/DubaiTechnologies/>
[Twitter] <https://twitter.com/DubaiTech1994> [LinkedIN]
<https://www.linkedin.com/in/dubai-technologies-b0884a125> [Instagram]
<https://www.instagram.com/dubaitechnologies/> [Instagram]
<https://www.youtube.com/channel/UCR5VDdt7lxlf12p6UX8u0sg>
