Re: Kafka queue full configuration

2017-03-27 Thread Manikumar
Regards, > Sudhir > > From: Mohapatra, Sudhir (Nokia - IN/Gurgaon) > Sent: Thursday, March 23, 2017 11:01 AM > To: 'users@kafka.apache.org' ; ' > d...@kafka.apache.org' > Subject: Kafka queue full configuration > > Hi, > We are trying to simulate the

RE: Kafka queue full configuration

2017-03-27 Thread Mohapatra, Sudhir (Nokia - IN/Gurgaon)
27; Subject: Kafka queue full configuration Hi, We are trying to simulate the kafka queue full scenarios on kafka 0.10.0. I have seen in earlier versions there is a configuration parameter "queue.buffering.max.messages" which can be set to simulate the queue full scenario. But in the

Kafka queue full configuration

2017-03-23 Thread Mohapatra, Sudhir (Nokia - IN/Gurgaon)
Hi, We are trying to simulate the kafka queue full scenarios on kafka 0.10.0. I have seen in earlier versions there is a configuration parameter "queue.buffering.max.messages" which can be set to simulate the queue full scenario. But in the kafka 0.10.0 this parameter is not th

Re: Kafka Queue

2016-12-07 Thread Hans Jespersen
are you setting the group.id property to be the same on both consumers? https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example -hans /** * Hans Jespersen, Principal Systems Engineer, Confluent Inc. * h...@confluent.io (650)924-2670 */ On Wed, Dec 7, 2016 at 12:36 PM, Justin

Re: Kafka Queue

2016-12-07 Thread Sharninder
Have you a set them to the same consumer group ID? That's what "identities" a consumer group. On Thu, Dec 8, 2016 at 2:06 AM, Justin Smith wrote: > I read this paragraph under Kafka as a Messaging System. > > > > “The consumer group concept in Kafka generalizes these two concepts. As > with a qu

Kafka Queue

2016-12-07 Thread Justin Smith
I read this paragraph under Kafka as a Messaging System. "The consumer group concept in Kafka generalizes these two concepts. As with a queue the consumer group allows you to divide up processing over a collection of processes (the members of the consumer group). As with publish-subscribe, Kafk

Re: Query on kafka queue message purge.

2016-10-14 Thread David Garcia
execute when the cleanup thread runs which, by default, is 5 mins. -David On 10/14/16, 3:09 AM, "Rudra Moharana" wrote: Hi Team, I need a help for my query Is there any way to remove the message from kafka queue with out stopping zookeeper or topic server

Query on kafka queue message purge.

2016-10-14 Thread Rudra Moharana
Hi Team, I need a help for my query Is there any way to remove the message from kafka queue with out stopping zookeeper or topic server or cluster. Thanks, Rudra

RE: can we send pdf, image files in kafka queue

2015-02-26 Thread Udbhav Agarwal
Thanks Joestein for the reply. I would be glad if I get some references of java code for the same. Secondly will it be a good approach to send complete files rather than the url of the files on kafka queue. Thanks, Udbhav Agarwal -Original Message- From: Joe Stein [mailto:joe.st

Re: can we send pdf, image files in kafka queue

2015-02-26 Thread Joe Stein
ng to write big messages. ~ Joestein On Feb 26, 2015 9:09 AM, "Udbhav Agarwal" wrote: > Hi, > Can we send pdf, image etc files in kafka queue. Not the url containing > the address of the pdf etc files but actualy pdf etc files. I want to send > pdf etc files from kafka produce

can we send pdf, image files in kafka queue

2015-02-26 Thread Udbhav Agarwal
Hi, Can we send pdf, image etc files in kafka queue. Not the url containing the address of the pdf etc files but actualy pdf etc files. I want to send pdf etc files from kafka producer to kafka consumer where I want to put the files in hdfs. Thanks, Udbhav Agarwal

Re: Clean Kafka Queue

2014-10-21 Thread Eduardo Costa Alfaia
-Harsha >> >> >> On Tue, Oct 21, 2014, at 08:00 AM, Eduardo Costa Alfaia wrote: >>> Hi Guys, >>> >>> Is there a manner of cleaning a kafka queue after that the consumer >>> consume the messages? >>> >>> Thanks >>> -- >>> Informativa sulla Privacy: http://www.unibs.it/node/8155 >> -- Informativa sulla Privacy: http://www.unibs.it/node/8155

Re: Clean Kafka Queue

2014-10-21 Thread Joe Stein
o delete a message after the consumer processed a message > there is no api for it. > -Harsha > > > On Tue, Oct 21, 2014, at 08:00 AM, Eduardo Costa Alfaia wrote: > > Hi Guys, > > > > Is there a manner of cleaning a kafka queue after that the consumer > > consum

Re: Clean Kafka Queue

2014-10-21 Thread Harsha
Costa Alfaia wrote: > Hi Guys, > > Is there a manner of cleaning a kafka queue after that the consumer > consume the messages? > > Thanks > -- > Informativa sulla Privacy: http://www.unibs.it/node/8155

Clean Kafka Queue

2014-10-21 Thread Eduardo Costa Alfaia
Hi Guys, Is there a manner of cleaning a kafka queue after that the consumer consume the messages? Thanks -- Informativa sulla Privacy: http://www.unibs.it/node/8155

Re: Kafka Queue Depth Metrics

2014-10-05 Thread Jun Rao
This is a jmx on the consumer side. The server side metric is for the internal replication consumer and is only available if you have a replication factor more than 1. Thanks, Jun On Fri, Oct 3, 2014 at 4:28 AM, Shah, Devang1 wrote: > I tried hooking up jconsole to Kafka server but could not s

Re: Kafka Queue Depth Metrics

2014-10-03 Thread Shah, Devang1
I tried hooking up jconsole to Kafka server but could not see the "kafka.consumer" MBean. I am using kafka_2.9.2-0.8.1.1 Am I going wrong anywhere ? Thanks, Devang

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Arnaud Lawson
t; > > Thanks, > > > > > > Jun > > > > > > > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson > > >wrote: > > > > > > > Hi, > > > > > > > > Does anyone know what

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Jun Rao
r that measures the # of messages a > > consumer is behind. > > > > Thanks, > > > > Jun > > > > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson > >wrote: > > > > > Hi, > > > > > > Does anyone know what the JMX m

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Arnaud Lawson
e a metric in the consumer that measures the # of messages a > consumer is behind. > > Thanks, > > Jun > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson >wrote: > > > Hi, > > > > Does anyone know what the JMX metric(s) are for determining the depth o

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Jun Rao
We also have a metric in the consumer that measures the # of messages a consumer is behind. Thanks, Jun On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson wrote: > Hi, > > Does anyone know what the JMX metric(s) are for determining the depth of > the kafka queue? I basically want

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Guozhang Wang
on certain topics. > > > > Guozhang > > > > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson > >wrote: > > > > > Hi, > > > > > > Does anyone know what the JMX metric(s) are for determining the depth > of > > > the kaf

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Chi Hoang
cified consumer on certain topics. > > > > Guozhang > > > > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson > >wrote: > > > > > Hi, > > > > > > Does anyone know what the JMX metric(s) are for determining the depth > of

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Arnaud Lawson
ffset of messages on the brokers v.s. the consumed last offsets > of the specified consumer on certain topics. > > Guozhang > > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson >wrote: > > > Hi, > > > > Does anyone know what the JMX metric(s) are for dete

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Guozhang Wang
. the consumed last offsets of the specified consumer on certain topics. Guozhang On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson wrote: > Hi, > > Does anyone know what the JMX metric(s) are for determining the depth of > the kafka queue? I basically want to know the amount of messa

Kafka Queue Depth Metrics

2014-03-31 Thread Arnaud Lawson
Hi, Does anyone know what the JMX metric(s) are for determining the depth of the kafka queue? I basically want to know the amount of messages that are on the queue at a certain point in time. Please let me know if there is a way to find that out. Thanks. -- Arnaud Lawson , Systems Operations

Re: java.lang.NumberFormatException: null - when producer tries to send message to Kafka queue

2013-06-07 Thread Andrea Gazzarini
partitions) associated. Cheers, Andrea On 06/07/2013 12:26 AM, Ashish Nigam wrote: Hi, I am getting exception when Producer tries to send messages to Kafka queue. It tries to connect to zookeeper instances and then throws exception. Here's initial log - 06 Jun 2013 15:17:29,541 [DEBU

Re: java.lang.NumberFormatException: null - when producer tries to send message to Kafka queue

2013-06-06 Thread Jun Rao
Which version of ZK are you using? Thanks, Jun On Thu, Jun 6, 2013 at 3:26 PM, Ashish Nigam wrote: > Hi, > I am getting exception when Producer tries to send messages to Kafka queue. > > It tries to connect to zookeeper instances and then throws exception. > Here's init

java.lang.NumberFormatException: null - when producer tries to send message to Kafka queue

2013-06-06 Thread Ashish Nigam
Hi, I am getting exception when Producer tries to send messages to Kafka queue. It tries to connect to zookeeper instances and then throws exception. Here's initial log - 06 Jun 2013 15:17:29,541 [DEBUG] ZkConnection | Creating new ZookKeeper instance to connect to 172.16.1.104