We are using
zookeeper-3.4.6
more zookeeper logs here:
2014-11-14 09:42:28,233 [myid:1] - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2182:NIOServerCnxn@1007] - Closed socket connection for
client /10.93.83.43:43487 which had sessionid 0x149a4cc1b581b60
2014-11-14 09:42:28,234 [myid:1] - WARN [N
Which version of ZK are you using?
Thanks,
Jun
On Thu, Nov 13, 2014 at 10:15 AM, Chen Wang
wrote:
> Thanks for the info.
> It makes sense, however, I didn't see any "session timeout"/"expired"
> entries in consumer log..
> but do see lots of connection closed entry in zookeeper log:
>
> 2014-1
>From your zk logs:
2014-11-13 10:07:53,132 [myid:1] - INFO [NIOServerCxn.Factory: .. Closed
socket connection for
2014-11-13 10:08:08,746 [myid:1] - INFO [NIOServerCxn.Factory: .. Closed
socket connection for
It does seem to kick out one consumer every 6ms, and you would probably
check you
kafka.config.zookeeper.session.timeout.ms6
kafka.config.rebalance.backoff.ms6000kafka.config.rebalance.max.retries6
On Thu, Nov 13, 2014 at 10:56 AM, Guozhang Wang wrote:
> I was originally asking about consumer configs, which should contain the
> following:
>
> http://kafka.apache.org/docum
I was originally asking about consumer configs, which should contain the
following:
http://kafka.apache.org/documentation.html#consumerconfigs
zookeeper.session.timeout.ms
zookeeper.connection.timeout.ms
On Thu, Nov 13, 2014 at 10:40 AM, Manish wrote:
> @Guozhang:
>
> In server.properties we
@Guozhang:
In server.properties we have :
zookeeper.connection.timeout.ms=100
In zoo.cfg we have
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zookeeper/data
dataLogDir=/opt/zookeeper/logs
clientPort=2182
server.1=.com:2888:3888
server.2=.com:2888:3888
server.3=.
Chen,
>From ZK logs it sounds like ZK kept timed out consumers which triggers
rebalance.
What is the zk session timeout config value in your consumers?
Guozhang
On Thu, Nov 13, 2014 at 10:15 AM, Chen Wang
wrote:
> Thanks for the info.
> It makes sense, however, I didn't see any "session timeo
@Neha, Can you share suggested consumer side GC settings?
Consumer side GC settings are not standard since it is a function of your
application that embeds the consumer. Your consumer application's memory
patterns will dictate your GC settings. Sorry, I know that's not very
helpful, but GC tuning
Thanks for the info.
It makes sense, however, I didn't see any "session timeout"/"expired"
entries in consumer log..
but do see lots of connection closed entry in zookeeper log:
2014-11-13 10:07:53,132 [myid:1] - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2182:NIOServerCnxn@1007] - Closed socket
Hey Chen,
As Neha suggested, typical reason of too many rebalances is that your
consumers kept being timed out from ZK, and you can verify this by checking
in your consumer logs for sth. like "session timeout" entries (these are
not ERROR entries).
Guozhang
Guozhang
On Wed, Nov 12, 2014 at 5:31
@Neha, Can you share suggested consumer side GC settings?
On Wed, Nov 12, 2014 at 5:31 PM, Neha Narkhede
wrote:
> Does this help?
>
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog
> ?
>
> On Wed, Nov 12, 2014 at 3:53 PM, Chen Wang
> wrote:
>
Does this help?
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog
?
On Wed, Nov 12, 2014 at 3:53 PM, Chen Wang
wrote:
> Hi there,
> My kafka client is reading a 3 partition topic from kafka with 3 threads
> distributed on different machines. I am
Hi there,
My kafka client is reading a 3 partition topic from kafka with 3 threads
distributed on different machines. I am seeing frequent owner changes on
the topics when running:
bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
my_test_group --topic mytopic -zkconnect localhost:21
13 matches
Mail list logo
