Help is processing huge data through Kafka-storm cluster

2014-06-14 Thread Shaikh Ahmed
Hi, Daily we are downloaded 28 Million of messages and Monthly it goes up to 800+ million. We want to process this amount of data through our kafka and storm cluster and would like to store in HBase cluster. We are targeting to process one month of data in one day. Is it possible? We have setup

Re: LeaderNotAvailableException in 0.8.1.1

2014-06-14 Thread Michael G. Noll
Have you given Kafka some time to re-elect a new leader for the "missing" partition when you re-try steps 1-5? See here: > If you do, you should be able to go through steps > 1-8 without seeing LeaderNotAvailableExceptions (you may need to give > Kafka some time to re-elect the remaining, second b