Hi Emmanuel,
You can firstly run a kafka producer perf (bin/kafka-producer-perf-test.sh)
test with your storm consumers and kafka consumer perf (bin/
kafka-consumer-perf.test.sh) test with your own producers respectively to
see if the bottleneck is really in kafka.
Thanks,
Manu Zhang
On Mon, Mar
Hi Emmanuel,
Can you post your kafka server.properties and in your producer are your
distributing your messages into all kafka topic partitions.
--
Harsha
On March 20, 2015 at 12:33:02 PM, Emmanuel (ele...@msn.com) wrote:
Kafka on test cluster:
2 Kafka nodes, 2GB, 2CPUs
3 Zookeeper no
Kafka on test cluster: 2 Kafka nodes, 2GB, 2CPUs3 Zookeeper nodes, 2GB, 2CPUs
Storm:3 nodes, 3CPUs each, on the same Zookeeper cluster as Kafka.
1 topic, 5 partitions, replication x2
Whether I use 1 slot for the Kafka Spout or 5 slots (=#partitions), the
throughput seems about the same.
I can't se
