Re: Getting None.get exception when running kafka-reassign-partitions.sh

2013-10-12 Thread Neha Narkhede
Can you please send the full stack trace? Thanks, Neha On Oct 12, 2013 1:56 PM, "Calvin Lei" wrote: > Checked the json file and everything seems normal. When I run the script > and I got the error: > > Partitions reassignment failed due to None.get > java.util.NoSuchElementException: None.get >

Getting None.get exception when running kafka-reassign-partitions.sh

2013-10-12 Thread Calvin Lei
Checked the json file and everything seems normal. When I run the script and I got the error: Partitions reassignment failed due to None.get java.util.NoSuchElementException: None.get at scala.None$.get (Option.scala:185) my json file: {partitions": [{"topic": "topicA", "partition": 0}] }

Re: Is there a way to pull out kafka metadata from zookeeper?

2013-10-12 Thread Neha Narkhede
What I suggested was *not* to get the metadata from the zookeeper, but simply the broker list, to avoid having a list brokers on the configuration. This seems reasonable for installations that don't have access to a VIP. One option is to support broker list and zookeeper and get the broker list fr

Re: I am finding a way to solve my problem (using kafka)

2013-10-12 Thread Guozhang Wang
Hello Guohao, Have you taken a look at Camus? https://groups.google.com/forum/#!forum/camus_etl Guozhang On Sat, Oct 12, 2013 at 2:42 AM, Zhao GuoHao(技术中心) < guohaozhao116...@sohu-inc.com> wrote: > Hello Kafka : > > I want to load message in brokers to hadoop,but I don't have a totally > de

I am finding a way to solve my problem (using kafka)

2013-10-12 Thread 技术中心
Hello Kafka : I want to load message in brokers to hadoop,but I don't have a totally design for it , Can you Give me a solution ? Use camps ? Hadoop consumer ? How to use it ? I find that the Hadoop Consumer in github (kafka version 0.7.2), I can only consume one partition of one topic ,but in

Re: Is there a way to pull out kafka metadata from zookeeper?

2013-10-12 Thread Bruno D. Rodrigues
What I understood from Neha is that querying the metadata from one (any) kafka will return everything in one go. Querying the data indirectly via zookeeper would be more complicated and would involve more requests between the zookeeper and the brokers before being able to answer back. What I su