Re: Kafka Metrics in 0.8

Yes, Kafka data shows up in the log dir in kafka-2. We are having 2 partitions on all brokers. On Mon, Jul 29, 2013 at 8:25 PM, Jun Rao wrote: > Does the Kafka data show up in the log dir in kafka-2? Since only the > leader can serve any traffic, if there is only have one partition or all > lea

Re: Leader not local for partition error?

Are you on Windows? We have seen issues like that before on Windows. You may have to use "/" when configuring "log.dirs". Thanks, Jun On Mon, Jul 29, 2013 at 4:50 PM, Maxime Petazzoni wrote: > Same issue with the 0.8 beta1 tarball. There is something interesting in > state-change.log though:

Re: Fatal issue (was RE: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress_

Hmm, it takes 90 secs to connect to ZK? That seems way too long. Is your ZK healthy. Also, are you on the 0.8 beta1 release? If not, could you try that one? It may not be related, but we did fix some consumer side deadlock issues there. Thanks, Jun On Mon, Jul 29, 2013 at 9:02 AM, Hargett, Phi

RE: Leader not local for partition error?

Same issue with the 0.8 beta1 tarball. There is something interesting in state-change.log though: [2013-07-29 16:47:26,708] TRACE Broker 0 received LeaderAndIsr request correlationId 6 from controller 0 epoch 1 starting the become-leader transition for partition [test,0] (state.change.logger) [

Re: Partitions per topic per broker?

Thanks for the clarification guys. -- Ian Friedman On Friday, July 26, 2013 at 1:16 AM, Jun Rao wrote: > I assume this is 0.7. In 0.7, partitions are per broker. So, if you > configure 100 partitions, each broker will have 100 partitions. In 0.8, > partitions is at the cluster level and won't

Re: Kafka Metrics in 0.8

Does the Kafka data show up in the log dir in kafka-2? Since only the leader can serve any traffic, if there is only have one partition or all leaders are on one broker, not all metrics (e.g., produce/fetch rate) will show up on both brokers. Thanks, Jun On Mon, Jul 29, 2013 at 6:04 AM, Hanish

Re: Fatal issue (was RE: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress_

Why would a consumer that has been shutdown still be rebalancing? Zookeeper session timeout (zookeeper.session.timeout.ms) is 1000 and sync time (zookeeper.sync.timeout.ms) is 500. Also, the timeout for the thread that looks for the leader is left at the default 200 milliseconds (refresh.leader

Re: Kafka Metrics in 0.8

Hi All, As kafka-0.8's main feature in data replication. We are running two kafka nodes(lets say: kafka-1 and kafka-2) in cluster mode. If we produce data into kafka using replication factor=2 then its replica is created on second machine. I am using jconsole to monitor kafka. if messages are go

RE: Client improvement discussion

In the past there was some discussion about having a C client for non-JVM languages. Is this still planned as well? Being able to work with Kafka from other languages would be a great thing. Where I work, we interact with Kafka via Java and Ruby (producer), so having an official C library tha

Re: Consumer stops consuming after 5 gb

Hi Neha, It turned out to be a Java Heap error. The try catch wasn't implemented properly. Regards, Samir On Tue, Jul 23, 2013 at 11:20 PM, Neha Narkhede wrote: > Samir, > > When you observe that the consumer hangs, could you send around the thread > dump. Also can you check the value of the F