Re: kafka brokers going down within 24 hrs

You can use tools (e.g., VisialVM) to diagnose OOM problem. 2015-01-15 14:15 GMT+08:00 Tousif Khazi : > i see this error > > ERROR [ReplicaFetcherThread-0-1], Error for partition > [realtimestreaming,1] to broker 1:class > kafka.common.NotLeaderForPartitionException > (kafka.server.ReplicaFetche

Re: Issues Running Kafka Producer Java example

Please check your consumer's position http://kafka.apache.org/documentation.html#basic_ops_consumer_lag 2014-11-16 7:09 GMT+08:00 Stream Processing : > anyone? > > On Sat, Nov 8, 2014 at 4:25 PM, Hardik Pandya > wrote: > > > Hello Champs, > > > > I am trying to run first java producer example

Re: Consumer reading from same topic twice without commiting

Auto Commit is to sync offset to zookeeper periodically. Without auto commit, consumer still maintains an offset locally. It explains this behavior. To be able to rewind, you need to use SimpleConsumer. 2014/11/15 上午2:18 於 "dinesh kumar" 寫道： > Hi, > I am trying to understand the behavior of Kafk

Re: One of two consumers is always Idle though I have 2 partitions

Hi Palur, When producing messages, did you specify a key in your KeyedMessage? If not, producer will send all messages to ONE randomly selected partition and stick to this partition for 10 minutes by default. regards, Chia-Chun 2014-11-14 7:19 GMT+08:00 Jagat Singh : > It would be worth reading

Re: consumer ack for high-level consumer?

it is considered as "consumed"; if you want > consumer to only consider a message as consumed when it is processed by the > application on top of it, you need to turn off auto offset and manually > call commit. > > On Thu, Nov 6, 2014 at 6:25 PM, Chia-Chun Shih > wrote: >

Re: One question about "New Producer Configs"

Dear Genlong, "New Producer Configs" is for upcoming versions. If you are using 0.8.1, please refer to "Producer Configs". acks=n, in which n means number of in-sync replicas, not number of partitions. regards, Chia-Chun 2014-11-10 10:51 GMT+08:00 hi <995174...@qq.com>: > Dear sir or madam, >

Re: consumer ack for high-level consumer?

: > That is correct. > > Guozhang > > On Wed, Nov 5, 2014 at 9:18 PM, Chia-Chun Shih > wrote: > > > Hi, > > > > Thanks for your response. I just read source code and found that: > > > > 1) ConsumerIterator$next() use PartitionTopicInfo$resetConsum

Re: consumer ack for high-level consumer?

sed > every single message but only a batch of messages. > > Guozhang > > On Tue, Nov 4, 2014 at 10:42 PM, Chia-Chun Shih > wrote: > > > Hi, > > > > I am a new to Kafka. In my understanding, high-level consumer ( > > ZookeeperConsumerConnector) changes offset

consumer ack for high-level consumer?

Hi, I am a new to Kafka. In my understanding, high-level consumer ( ZookeeperConsumerConnector) changes offset when message is drawn by ConsumerIterator. But I would like to change offset when message is processed, not when message is drawn from broker. So if a consumer dies before a message is co