Re: Cleanup unused topics

2018-08-21 Thread Brett Rann
These are in regex form for DataDog's JMX collector, but it should get you started: bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=TotalProduceRequestsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=TotalFetchRequestsPerSec,topic=.*' bean_r

Cleanup unused topics

2018-08-21 Thread Luigi Tagliamonte
I've got a sandbox cluster (without any control on topic creation) full of unused topics and I'd like to cleanup all the those topics. I was able to identify all the topics with 0 messages with the following lines: *#!/bin/bash* *set -e* *BROKERS="10.16.208.220:9092 ,10.

Kakfa embedded cluster rebalance scenario test

2018-08-21 Thread pradeep s
Hi , I would like to add a integration test for kafka rebalance scenario and make sure retries are working as expected if enough replicas are not there in my kafka streams application . Do you have any info on how can i achieve this . I was checking TestUtils class in org.apache.kafka.test, but co

Re: Improve error message when trying to produce message without key for compacted topic

2018-08-21 Thread Matthias J. Sax
That makes sense to me. Please file a Jira. Thanks a lot for reporting this issue! -Matthias On 8/21/18 3:29 AM, Patrik Kleindl wrote: > Hello > > Yesterday we had the following exception: > > Exception thrown when sending a message with key='null' and payload='...' > to topic sometopic:: org

Improve error message when trying to produce message without key for compacted topic

2018-08-21 Thread Patrik Kleindl
Hello Yesterday we had the following exception: Exception thrown when sending a message with key='null' and payload='...' to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: This message has failed its CRC checksum, exceeds the valid size, or is otherwise corrupt. The cau

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

2018-08-21 Thread Viktor Somogyi-Vass
Congrats Dong! :) On Tue, Aug 21, 2018 at 10:09 AM James Cheng wrote: > Congrats Dong! > > -James > > > On Aug 20, 2018, at 3:54 AM, Ismael Juma wrote: > > > > Hi everyone, > > > > Dong Lin became a committer in March 2018. Since then, he has remained > > active in the community and contributed

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

2018-08-21 Thread James Cheng
Congrats Dong! -James > On Aug 20, 2018, at 3:54 AM, Ismael Juma wrote: > > Hi everyone, > > Dong Lin became a committer in March 2018. Since then, he has remained > active in the community and contributed a number of patches, reviewed > several pull requests and participated in numerous KIP d

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

2018-08-21 Thread Dong Lin
Thank you everyone!! It has been my great pleasure to contribute to and be part of the Apache Kafka community! On Mon, Aug 20, 2018 at 3:50 PM, Mayuresh Gharat wrote: > Congrats Dong !!! > > Thanks, > > Mayuresh > > On Mon, Aug 20, 2018 at 1:36 PM Gwen Shapira wrote: > > > Congrats Dong Lin! W