Is that a single node, local host only installation?
For starters, I'd advice you to use the --new-consumer option - it will use
what I think is a way less hairy method of doing consumer groups. W.r.t.
Zookeeper, I've found that in test installations, I often end up wiping its
data and restarting,
I've also asked on stackoverflow, in case you prefer to answer there:
http://stackoverflow.com/questions/36313470/consumerrebalancefailedexception-with-the-kafka-console-consumer
Thanks,
Filipe
On Wed, Mar 30, 2016 at 4:03 PM, Filipe Correia
wrote:
> Hi there,
>
> I've just installed kafka 0.9.
rConsumerConnector.scala:326)
> [06/02/14 08:59:18:018 AM EST] 215 INFO mkconsumer.MKConsumer: Put to
> table Credit.SMC.MUNI info put gdm entity Credit.SMC.MUNI, key
> MUNI.SMC.134103391 takes 4 ms
> [06/02/14 08:59:18:018 AM EST] 215 INFO mkconsumer.MKConsumer
25, 2014 4:03 PM
To: users@kafka.apache.org
Subject: Re: ConsumerRebalanceFailedException
Could you send around the consumer log when it throws
ConsumerRebalanceFailedException. It should state the reason for the failed
rebalance attempts.
Thanks,
Neha
On Tue, Feb 25, 2014 at 12:01 PM, Yu, Li
Could you send around the consumer log when it throws
ConsumerRebalanceFailedException. It should state the reason for the failed
rebalance attempts.
Thanks,
Neha
On Tue, Feb 25, 2014 at 12:01 PM, Yu, Libo wrote:
> Hi all,
>
> I tried to reproduce this exception. In case one, when no broker wa
wrote:
> > >
> > > > Hi Jun,
> > > >
> > > > zookeeper.session.timeout.ms is used in a broker's configuration and
> > > > manages brokers' registration with zk.
> > > > Does it apply to consumer as well? Thanks.
> > > &
, Yu, Libo wrote:
> >
> > > Hi Jun,
> > >
> > > zookeeper.session.timeout.ms is used in a broker's configuration and
> > > manages brokers' registration with zk.
> > > Does it apply to consumer as well? Thanks.
> > &g
t; >
> > > zookeeper.session.timeout.ms is used in a broker's configuration and
> > > manages brokers' registration with zk.
> > > Does it apply to consumer as well? Thanks.
> > >
> > > Regards,
> > >
> > > Libo
> >
th zk.
> > Does it apply to consumer as well? Thanks.
> >
> > Regards,
> >
> > Libo
> >
> >
> > -Original Message-
> > From: Jun Rao [mailto:jun...@gmail.com]
> > Sent: Monday, December 30, 2013 11:13 AM
> > To: u
s,
>
> Libo
>
>
> -Original Message-
> From: Jun Rao [mailto:jun...@gmail.com]
> Sent: Monday, December 30, 2013 11:13 AM
> To: users@kafka.apache.org
> Subject: Re: ConsumerRebalanceFailedException
>
> If the consumer is not shut down properly, it
To: users@kafka.apache.org
Subject: Re: ConsumerRebalanceFailedException
If the consumer is not shut down properly, it will take
zookeeper.session.timeout.ms before the consumer is deregistered from ZK.
If you restart the consumer before that, rebalances may fail.
Make sure tha
If the consumer is not shut down properly, it will take
zookeeper.session.timeout.ms before the consumer is deregistered from ZK.
If you restart the consumer before that, rebalances may fail.
Make sure that you call connector.shutdown() when you shut down the consumer
Thanks,
Jun
On Mon, Dec 3
n. How long is the interval between the retries?
> Is it max.fetch.wait + socket.timeout.ms?
> Thanks.
>
> Libo
>
>
> -Original Message-
> From: Jun Rao [mailto:jun...@gmail.com]
> Sent: Monday, December 02, 2013 11:55 AM
> To: users@kafka.apache.org
> Subject
Message-
From: Jun Rao [mailto:jun...@gmail.com]
Sent: Monday, December 02, 2013 11:55 AM
To: users@kafka.apache.org
Subject: Re: ConsumerRebalanceFailedException
Is the failure on the last rebalance? If so, some partitions will not have any
consumers. A common reason for rebalance failure is that
: Friday, November 29, 2013 8:50 PM
> To: users@kafka.apache.org
> Subject: Re: ConsumerRebalanceFailedException
>
> Transient rebalance failures are ok. However, it's important that the last
> rebalance in a sequence succeeds. Otherwise, some of the partitions will
> not be consumed by
n Rao [mailto:jun...@gmail.com]
Sent: Friday, November 29, 2013 8:50 PM
To: users@kafka.apache.org
Subject: Re: ConsumerRebalanceFailedException
Transient rebalance failures are ok. However, it's important that the last
rebalance in a sequence succeeds. Otherwise, some of the partitions w
ZKRebalancerListener$$anon$1.r
> un(ZookeeperConsumerConnector.scala:326)
>
> Regards,
>
> Libo
>
>
> -Original Message-
> From: Joe Stein [mailto:joe.st...@stealth.ly]
> Sent: Friday, November 29, 2013 11:57 AM
> To: users@kafka.apache.org
> Subject: Re
istener$$anon$1.r
un(ZookeeperConsumerConnector.scala:326)
Regards,
Libo
-Original Message-
From: Joe Stein [mailto:joe.st...@stealth.ly]
Sent: Friday, November 29, 2013 11:57 AM
To: users@kafka.apache.org
Subject: Re: ConsumerRebalanceFailedException
What is the full stack trace? if you see "can't r
tion?
>
> Regards,
>
> Libo
>
>
> -Original Message-
> From: Florin Trofin [mailto:ftro...@adobe.com]
> Sent: Tuesday, July 16, 2013 4:20 PM
> To: users@kafka.apache.org
> Subject: Re: ConsumerRebalanceFailedException
>
> Yes, I think these are two se
We found our consumer stopped working after this exception occurred.
Can the consumer recover from such an exception?
Regards,
Libo
-Original Message-
From: Florin Trofin [mailto:ftro...@adobe.com]
Sent: Tuesday, July 16, 2013 4:20 PM
To: users@kafka.apache.org
Subject: Re
Yes, I think these are two separate issues.
F.
On 7/16/13 11:32 AM, "Joel Koshy" wrote:
>From a user's perspective, ConsumerRebalanceException is a bit cryptic
>-I think the other thread was to provide a more informative message
>and also be able to recover when a broker does come up (fixed in
>From a user's perspective, ConsumerRebalanceException is a bit cryptic
-I think the other thread was to provide a more informative message
and also be able to recover when a broker does come up (fixed in
KAFKA-969).
Thanks,
Joel
On Tue, Jul 16, 2013 at 11:04 AM, Vaibhav Puranik wrote:
> Thank
Thank you Joel.
In a different but related thread, somebody is asking to rename the
exception as NoBrokerAvailableExcption. But given the description above,
the exception seems to be named appropriately.
Regards,
Vaibhav
On Tue, Jul 16, 2013 at 12:05 AM, Joel Koshy wrote:
> Yes - rebalance =>
Yes - rebalance => consumers trying to coordinate through ZK.
Rebalances can happen when one or more of the following happen:
- a consumed topic partition appears or disappears - i.e., if a broker
comes or goes.
- a consumer instance in the group comes or goes
"goes" could also be triggered by sess
I think this is addressed in KAFKA-969
On Fri, Jul 12, 2013 at 5:55 PM, Florin Trofin wrote:
> Hello,
>
> I'm going through a series of tests with Kafka 0.8 to see what the failure
> cases are. One simple scenario is this:
>
> 1. Launch Zookeeper
> 2. Launch my consumer (written in scala) but do
I have seen those exceptions when you restart your Kafka client before the
configured Zookeeper session timeout has occured.
Try and wait a bit longer between client restarts (we wait for 40s) and see
if those exceptions go away.
On Tuesday, March 12, 2013, Dragos Manolescu wrote:
> Hi --
>
> I h
Do you see many rebalances in your consumer log?
Thanks,
Jun
On Tue, Mar 12, 2013 at 6:01 PM, Dragos Manolescu <
dragos.manole...@servicenow.com> wrote:
> Hu Jun --
>
> I appreciate the fast turnaround. I use Zookeeper 3.4.5.
>
> I am not sure about the consumer GC: the exception happens when I
Hu Jun --
I appreciate the fast turnaround. I use Zookeeper 3.4.5.
I am not sure about the consumer GC: the exception happens when I fire up
the client's VM which shouldn't be a busy time for the GC--I've been using
autooffset.reset=largest so I assume that the client doesn't do a whole
lot upon
What's the ZK version? You should use 3.3.4 or above, which fixed some bugs
that could cause rebalance to fail.
Also, a typical reason for many rebalancing is the consumer side GC. If
so, you will see logs saying sth like "expired session" for ZK.
Thanks,
Jun
On Tue, Mar 12, 2013 at 9:47 AM,
29 matches
Mail list logo
