Question on Kafka .8 / Logstash client

2014-06-12 Thread Sutanu Das
Hi Kafka Users, 1. Is there a plugin/client for Logstash for Kakfa 0.8 ? 2. Is there any example of using Kafka using Logstash? I'm not in the mailing list yet so I'll really appreciate a email response - thanks for all your contribution to Kafkas project

Kafka High Level Consumer Fail Over

2014-06-12 Thread Bhavesh Mistry
Hi Kafka Dev Team/ Users, We have high level consumer group consuming from 32 partitions for a topic. We have been running 48 consumers in this group across multiple servers. We have kept 16 as back-up consumers, and hoping when the consumer dies, meaning when Zookeeper does not have an owner

Re: scala based service/deamon consumer example

2014-06-12 Thread S Ahmed
ah ok, " Kafka that may block if there are no new messages available." So the iterator blocks thanks. On Thu, Jun 12, 2014 at 4:13 PM, Guozhang Wang wrote: > You may want to read this wiki: > > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example > > Guozhang > > > On Thu,

Re: scala based service/deamon consumer example

2014-06-12 Thread Guozhang Wang
You may want to read this wiki: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example Guozhang On Thu, Jun 12, 2014 at 11:13 AM, S Ahmed wrote: > Is there a simple example (scala preferred) where there is a consumer that > is written to run as a deamon i.e. it keeps as open

[ANN] New Clojure library for Kafka

2014-06-12 Thread István
Hi all, I was working with Kafka the other day and I wanted to use my favorite language but the "official" driver was rather cryptic and there too much black magic going on, so I decided to rewrite it (violating the No. 1 rule of software development from Joel Spolsky). Anyways, it is short and mo

Re: LeaderNotAvailableException in 0.8.1.1

2014-06-12 Thread Prakash Gowri Shankor
So if we go back to the 2 broker case, I tried your suggestion with replication-factor 2 ./kafka-topics.sh --topic test2 --create --partitions 3 --zookeeper localhost:2181 --replication-factor When i repeat steps 1-5 i still see the exception. When i go to step 8 ( back to 2 brokers ), I dont s

scala based service/deamon consumer example

2014-06-12 Thread S Ahmed
Is there a simple example (scala preferred) where there is a consumer that is written to run as a deamon i.e. it keeps as open connection into a broker and reads off new messages

[Issue] Cannot connect to Kafka cluster on GCE from local machine

2014-06-12 Thread 傅駿浩
Hi all, I want to use the KafkaSpout(where is https://github.com/apache/incubator-storm/tree/master/external/storm-kafka) to read data from kafka. But my kafka and zookeeper cluster are builted on GCE(google cloud engine) and my storm testing is on local machine. The problem is I cannot set pu

Re: Could you pls add my email to the mailing list

2014-06-12 Thread François Langelier
- users@kafka.apache.org < users@kafka.apache.org>: A list for general user questions about Kafka. To subscribe, send an email to users-subscr...@kafka.apache.org < users-subscr...@kafka.apache.org>. Archives are available here. *Source: https://kafka.apache.org/contact.html Françoi