I want to rebuild my message queue from based on Redis to Kafka (because Redis persistence is terrible).
Now I am using the new KafkaProducer, and I am wondering which consumer should I use. Currently there are 3 Kafka Consumers (until 0.8.2.1): 1. High Level Consumer 2. Simple Consumer 3. KafkaConsumer (new introduced since 0.8) The High Level Consumer API looks pretty simple. It does a lot of complex work like failure handling and partitions rebalancing underneath user’s codes. This is great and I suppose it’s robust enough because I have already saw some projects using it. However, I can do little besides poll messages and commit offsets, so sometimes I feel that my hands are tied. The Simple Consumer, on the other side, offers too much options over consumers control, which make it hard to write consumer codes. There’s a new KafkaConsumer in the new Kafka clients maven release: http://kafka.apache.org/083/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html <http://kafka.apache.org/083/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html> The new KafkaConsumer seems very moderate, it offers more control options over High Level Consumer, and hides some details than the Simple Consumer. So I want to know is this KafkaConsumer ready for production use? Or will it change it’s API in the future? And why is there no documents on this new consumer in the Kafka’s main document page? Is it not robust enough? Thanks in advance!
