Re: callback handler is not getting called if cluster is down

ing? > > Currently if a partition is offline, the message in producer will not be > sent but sit in accumulator until the partition comes back online. Do you > mean you want to use the message send callback to detect broker failure? > > Jiangjie (Becket) Qin > > On 6/8/15, 12:

callback handler is not getting called if cluster is down

Hi, we are using .8.2.0 broker and default async producer to send the message*. we recently found out that if whole cluster gets down then callback handler is not getting called while we are getting below exception continuously* *12:36:41,267# WARN [Selector] - Error in I/O with localhost/127.

Re: New Java Producer Client handling unreachable Kafka

Hi Samuel, you can use *metadata.fetch.timeout.ms *property to reduce the blocking time while cluster is unreachable. The default value of this property is 1min. *waitOnMetadata*() is blocking call which blocks current thread waiting for the metadata fetch to

Re: Kafka Sync Producer threads(ack=0) are blocked

any insight would be useful. On Tue, Mar 24, 2015 at 11:06 AM, ankit tyagi wrote: > Hi All, > > Currently we are using kafka_2.8.0-0.8.0-beta1 in our production system. I > am using sync producer with ack=0 to send the events to broker. > > but I am seeing most of my pr

Kafka Sync Producer threads(ack=0) are blocked

Hi All, Currently we are using kafka_2.8.0-0.8.0-beta1 in our production system. I am using sync producer with ack=0 to send the events to broker. but I am seeing most of my producer threads are blocked. "jmsListnerTaskExecutor-818" prio=10 tid=0x7f3f5c05a800 nid=0x1719 waiting for monitor

Documentation is not proper for providing custom partitioner new Producer in http://kafka.apache.org/documentation.html

Hi, I have been going through http://kafka.apache.org/documentation.html and read below for providing custom partitioner. - provides software load balancing through an optionally user-specified Partitioner - The routing decision is influenced by the kafka.producer.Partitioner. inte

High Level Consumer Example in 0.8.2

Hi All, we are upgrading our kafka client version from 0.8.0 to 0.8.2. Is there any document for High level kafka consumer withMultiple thread like https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example for this newer version.

Re: High Level Consumer Example in 0.8.2

ould work for you: > > http://search.maven.org/#artifactdetails|org.apache.kafka|kafka_2.10|0.8.2.1|jar > > By the way, you probably also want 0.8.2.1 now since it fixed a few > important bugs in 0.8.2.0. > > -Ewen > > On Wed, Mar 11, 2015 at 10:08 PM, ankit tyagi > wrote:

Re: High Level Consumer Example in 0.8.2

> > There is a new high level consumer on the way and an initial version has > been checked into trunk, but it won't be ready to use until 0.9. > > > On Wed, Mar 11, 2015 at 9:05 AM, ankit tyagi > wrote: > > > Hi All, > > > > we are upgrading our ka

custom serializer and deserializer

Hi, I want to use protobuff for serializing and deserializing kafkaevents in .8.2.0. I can provide my custom serializer by setting KEY_SERIALIZER_CLASS_CONFIG and VALUE_SERIALIZER_CLASS_CONFIG. but how can I provide custom Deserializer.

Re: CallBackHandler is not being called after successful delivery of message

should still > see all the callbacks fire. > > -Jay > > On Mon, Feb 16, 2015 at 12:11 PM, ankit tyagi > wrote: > > > Hey, > > > > I am doing POC on kafka .8.2.0 version.Currently I am using kafka-client > > of 0.8.2.0 version for producing async m

CallBackHandler is not being called after successful delivery of message

Hey, I am doing POC on kafka .8.2.0 version.Currently I am using kafka-client of 0.8.2.0 version for producing async message with callBackHandler. I am using batch.size =1 in my producer cleint. As per my Understanding , This should behave like a sync client though message can be published from d

Re: Potential socket leak in kafka sync producer

> -Jaikiran > > > On Friday 30 January 2015 01:08 PM, ankit tyagi wrote: > >> I have shared object histogram after and before gc on gist >> https://gist.github.com/ankit1987/f4a04a1350fdd609096d >> >> On Fri, Jan 30, 2015 at 12:43 PM, Jaikiran Pai >> wrote

Re: Potential socket leak in kafka sync producer

c: {}", myTopic);* *if (myProducer != null) {* *myProducer.close();* *}* *}* } On Fri, Jan 30, 2015 at 1:08 PM, ankit tyagi wrote: > I have shared object histogram after and before gc on gist > https://gist.github.com/ankit1987/f4a04a1350fdd609096

Re: Potential socket leak in kafka sync producer

s. > > > -Jaikiran > > On Friday 30 January 2015 12:20 PM, ankit tyagi wrote: > >> Hi, >> >> I am closing my producer at the time of shutting down my application. >> >> @PreDestroy >> public void stop() >> { >> LO

Re: Potential socket leak in kafka sync producer

2015 at 11:22 AM, Manikumar Reddy wrote: > Hope you are closing the producers. can you share the attachment through > gist/patebin > > On Fri, Jan 30, 2015 at 11:11 AM, ankit tyagi > wrote: > > > Hi Jaikiran, > > > > I am using ubuntu and was able to repro

Re: Potential socket leak in kafka sync producer

code > which shows how you are using the Kafka APIs? > > -Jaikiran > On Thursday 29 January 2015 04:36 PM, ankit tyagi wrote: > >> Hi, >> >> Currently we are using sync producer client of 0.8.1 version in our >> production box . we are getting the fo

Potential socket leak in kafka sync producer

Hi, Currently we are using sync producer client of 0.8.1 version in our production box . we are getting the following exception while publishing kafka message *[2015-01-29 13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89] Fetching topic metadata with correlation id 10808 for topics

Re: too many rebalances in my consumer log and fetcher threads are getting stopped

Can i reduce the consumer balances if i increase the zookeeper session timeout? On Sat, Apr 19, 2014 at 9:11 PM, ankit tyagi wrote: > Hi Jun, > > i grabbed for 'begin rebalancing consumer' in my log and got around > *5000*times in today's logs. > > Can there

Re: too many rebalances in my consumer log and fetcher threads are getting stopped

:31 AM, Jun Rao wrote: > Do you see any rebalances? The fetcher was stopped because it was shut > down, which typically happens during rebalances. > > Thanks, > > Jun > > > On Fri, Apr 18, 2014 at 11:08 AM, ankit tyagi >wrote: > > > I have checked that. There

Re: too many rebalances in my consumer log and fetcher threads are getting stopped

splay/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog > ? > > Thanks, > > Jun > > > On Fri, Apr 18, 2014 at 3:58 AM, ankit tyagi >wrote: > > > Hi, > > > > I am seeing consumer re-balances very frequently and getting socket > > reconnec

too many rebalances in my consumer log and fetcher threads are getting stopped

16.33 50.87 89.83 1143 841.276 105 1129.520 1970.796 0.00 2.68 63.77 50.87 89.87 1143 841.276 105 1129.520 1970.796 Not sure what is happening over here. any leads would be helpful.. Regards, Ankit TYagi

Re: kafka consumer threads are in waiting state and lag is getting increased

inite*. On Tue, Apr 15, 2014 at 12:34 PM, ankit tyagi wrote: > I have increased the partition for parallelism but my concern is , if > message are present in partition then why consumer thread are blocked at > below condition. > > *KafkaConsumer-24" prio=10 tid=0x7f6da5

Re: kafka consumer threads are in waiting state and lag is getting increased

; Hi, > > Can you please check weather this is the situation > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped,why? > > Arjun Kota > > > On Tuesday 15 April 2014 11:49 AM, ankit tyagi wrote: > > Hi. > > currently w

kafka consumer threads are in waiting state and lag is getting increased

Hi. currently we are using *kafka_2.8.0-0.8.0-beta1 and *and high level consumer group to consume messages. Topic has been created with 3 replica and 100 partition so that max 100 threads can consume messages simultaneously, but i am seeing that mostly threads are in waiting state and lag is getti