Re: Producer does not recognize new brokers

2015-04-13 Thread Chi Hoang
I highly recommend https://github.com/airbnb/kafkat, which will simplify your partition management tasks. Use it with https://github.com/airbnb/kafkat/pull/3 for partition specific reassignment. Chi On Mon, Apr 13, 2015 at 4:08 AM, Jan Filipiak wrote: > Hey, > > try to not have newlines \n in

Re: Producer does not recognize new brokers

2015-04-13 Thread Jan Filipiak
Hey, try to not have newlines \n in your jsonfile. I think the parser dies on those and then claims the file is empty Best Jan On 13.04.2015 12:06, Ashutosh Kumar wrote: Probably you should first try to generate proposed plan using --generate option and then edit that if needed. thanks

Re: Producer does not recognize new brokers

2015-04-13 Thread Ashutosh Kumar
Probably you should first try to generate proposed plan using --generate option and then edit that if needed. thanks On Mon, Apr 13, 2015 at 3:12 PM, shadyxu wrote: > Thanks guys. You are right and then here comes another problems: > > I added new brokers 4, 5 and 6. Now I want to move partitio

Re: Producer does not recognize new brokers

2015-04-13 Thread shadyxu
Thanks guys. You are right and then here comes another problems: I added new brokers 4, 5 and 6. Now I want to move partitions 3, 4 and 5(currently on broker 1, 2 and 3) of topic test to these brokers. I wrote r.json file like this: {"partitions": [{"topic": "test","partition": 3,"replicas": [4]}

Re: Producer does not recognize new brokers

2015-04-12 Thread Ashutosh Kumar
I think you need to re balance the cluster. something like bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --topics-to-move-json-file topics-to-move.json --broker-list "5,6" --generate On Mon, Apr 13, 2015 at 11:22 AM, shadyxu wrote: > I added several new brokers to the cluster, th

Re: Producer does not recognize new brokers

2015-04-12 Thread Madhukar Bharti
Hi, Simply adding the brokers to the cluster will not reassign or redistribute topic partitions to newly added brokers. As it is also mentioned in documentation. "*However these new servers will not automatically be assigned any data partitions, so unless partitions are moved to them they won't b

Producer does not recognize new brokers

2015-04-12 Thread shadyxu
I added several new brokers to the cluster, there should'v been a rebalance but it seemed that the producer was not aware of the new brokers. Data kept being sent to the old brokers and there were no partitions on the new brokers. I configured the old brokers to the producer and did not restart th