I suppose you can either use the High Level Consumer or the simple consumer as per your use case. Simple Consumer does not do offset management and you will have to do it yourself but provides more control. High level consumer on the other hand does the offset management for you. The new Kafka Consumer is not yet 100 % ready I suppose. I would suggest using the High Level Cosnumer.
Thanks, Mayuresh On Mon, May 18, 2015 at 9:10 PM, luo.fucong <bayinam...@gmail.com> wrote: > I want to rebuild my message queue from based on Redis to Kafka (because > Redis persistence is terrible). > > Now I am using the new KafkaProducer, and I am wondering which consumer > should I use. > > Currently there are 3 Kafka Consumers (until 0.8.2.1): > > 1. High Level Consumer > 2. Simple Consumer > 3. KafkaConsumer (new introduced since 0.8) > > The High Level Consumer API looks pretty simple. It does a lot of complex > work like failure handling and partitions rebalancing underneath user’s > codes. This is great and I suppose it’s robust enough because I have > already saw some projects using it. However, I can do little besides poll > messages and commit offsets, so sometimes I feel that my hands are tied. > > The Simple Consumer, on the other side, offers too much options over > consumers control, which make it hard to write consumer codes. > > There’s a new KafkaConsumer in the new Kafka clients maven release: > http://kafka.apache.org/083/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html > < > http://kafka.apache.org/083/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html > > > The new KafkaConsumer seems very moderate, it offers more control options > over High Level Consumer, and hides some details than the Simple Consumer. > So I want to know is this KafkaConsumer ready for production use? Or will > it change it’s API in the future? And why is there no documents on this new > consumer in the Kafka’s main document page? Is it not robust enough? > > Thanks in advance! > -- -Regards, Mayuresh R. Gharat (862) 250-7125
