Zookeeper Connection Error-zxid

2015-09-16 Thread rajashree.khare
Hi, We are just starting to use Kafka. And we are trying to accomplish a very basic setup. Single Zookeeper with 3 Kafka brokers all on the same server. Initially this setup worked well and we were able to run our application. Our application ran into a recursive error and the log files kept app

Re: topics, partitions, consumergroups

2015-09-16 Thread tao xiao
Joris, You checkout Burrow https://github.com/linkedin/Burrow which gives you monitoring and alerting capability for offset stored in Kafka On Wed, 16 Sep 2015 at 23:05 Joris Peeters wrote: > I intend to write some bespoke monitoring for our internal kafka system. > Amongst others, it should gi

Re: Thread safety when committing from multiple threads

2015-09-16 Thread Ashish Shenoy
Hi, Can anyone please shed some light on this ? I couldn't find any pointers online other than this: http://stackoverflow.com/questions/26477585/when-calling-commitoffsets-on-kafka-high-level-consumerconnector-what-value-get Thanks, Ashish On Mon, Sep 14, 2015 at 2:06 PM, Ashish Shenoy wrote:

Why my brokers delete the whole partition data(150G) in one second, and recover later by replication?

2015-09-16 Thread Yiming Liu
Hello, Kafka Expert: I met some strange problem recently. Some topic may delete the whole partition data in one second, and recover later by replication. The the example of topic "ubt.mobilemonitor". It has 30 partitions, with replication 2. The retention.bytes is 157374182400(150G). The default s

topics, partitions, consumergroups

2015-09-16 Thread Joris Peeters
I intend to write some bespoke monitoring for our internal kafka system. Amongst others, it should give us warning if some topics are lagging behind too much (i.e. production much faster than consumption). What would currently be the neatest way to get a list of all topics, partitions and consu

Re: Jumbled up ISR

2015-09-16 Thread Prabhjot Bharaj
Thanks for your inputs Grant ! Regards, Prabhjot On Tue, Sep 15, 2015 at 6:56 PM, Grant Henke wrote: > The first replica in the ISR is the preferred replica, but is not required > to be the leader at all times. If you execute a preferred leader election, > or enable auto.leader.rebalance.enable