Re: Getting my Kafka code example working out of the box

2013-11-10 Thread S L
Hi, I was trying the single producer/consumer model via http://kafka.apache.org/documentation.html#quick-start. The code was from https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Examplefor the producer. I'll try to repeat all the steps again in case I missed something but this wa

Re: Getting my Kafka code example working out of the box

2013-11-10 Thread S L
Yes, I have 1 consumer running with 1 partition; or at least i think i have 1 partition. I ran the cmd from the documentation to create the topic *bin/kafka-create-topic.sh --zookeeper localhost:2181 --replica 1 --partition 1 --topic test* and I ran the console consumer command later, as specifi

Re: Getting my Kafka code example working out of the box

2013-11-10 Thread Priya Matpadi
If you have 5 partitions and 3 consumers in one consumer group, the consumers will balance all 5 partitions such that 2 consumers will get data from 2 partitions each and 1 consumer will get data from remaining 5th partition. If you have 3 partitions and 5 consumers in one consumer group, 3 consume

Re: Getting my Kafka code example working out of the box

2013-11-10 Thread Joe Stein
Do you have enough consumers representing the number of partitions you have? You should have one consumer within a consumer group for each partition... Think of it as threads, if you have 5 partitions you need 5 threads... if you only have 3 consumers(threads) then it is possible the data is on on

Re: Getting my Kafka code example working out of the box

2013-11-10 Thread S L
Thanks for everyone's reply and trying to help me. I think I got the test producer program to work but I don't know why. Again, using the test code found at https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Examplefor the producer straight from the website didn't work. However, wh

Re: Getting my Kafka code example working out of the box

2013-11-08 Thread Chris Bedford
I had some trouble with maven dependencies when i tried to get a simple round trip test going. I worked past those and made my test available here: https://github.com/buildlackey/cep/tree/master/kafka it should run out of the box. -cb On Thu, Nov 7, 2013 at 5:35 PM, S L wrote: > Hi, > > Th

Re: Getting my Kafka code example working out of the box

2013-11-07 Thread Joe Stein
It might be you are starting a consumer after the messages are produced and since your consumer is starting for the first time when it registers with ZK you won't see messages since the default is to start at the largest offset. so, try starting ZK, the broker and your consumer and then start up t

Getting my Kafka code example working out of the box

2013-11-07 Thread S L
Hi, This might be a really, really simple question but how do I get my test Kafka program working out of the box? I followed the directions from http://kafka.apache.org/documentation.html#quickstart. I started zk, the server, the producer and consumer. I played with the producer, sending msgs t