KafkaConsumer poll poor performance (0.10.0.0)

2016-10-04 Thread Mudassir Maredia
I am using KafkaConsumer from 0.10.0.0 version. Below is the sample code. I have tried to optimize it to get to a maximum number but that number is really very small. At times I have seen 80,000 records processed per second but at times it shows only 10,000 records per second. I have tried differen

list of stream or consumer objects per topic

2016-09-30 Thread Mudassir Maredia
I am using kafka api 0.10. //Sample code List topicsList = new ArrayList<>(); topicsList.add("topic1"); topicsList.add("topic2"); KafkaConsumer consumer = new KafkaConsumer(props); consumer.subscribe(topicsList); Problem: For each topic, I w