Re: Producer waiting ~15 mins before disconnecting.

2015-05-19 Thread 4mayank
Thanks Magnus. In this case the connections are not idle. There is active traffic between the producer/client and the kafka node when the node goes down. There are socket timeouts arguments for SimpleConsumer. But there are none when creating the producer. If there a configuration/poroperty item to

Re: Producer waiting ~15 mins before disconnecting.

2015-05-19 Thread Magnus Edenhill
Hi Mayank, The client should expose a configuration property to enable TCP keepalives (SO_KEEPALIVE) on its broker sockets, SO_KEEPALIVE provides speedier detection of connection loss on idle connections. (as a positive side effect it also helps keeping connections alive through NAT/firewalls/LBs)

Producer waiting ~15 mins before disconnecting.

2015-05-19 Thread 4mayank
I am using kafka 0.8.2.1 old producer. When one of the kafka node in the remote cluster is down the producer is waiting about 15 minutes before it disconnects and tries to connect to another node. (kafka takes < 1 min to change leaders). Producer config used: request.required.acks=1 partitioner.cl