Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-18 Thread Alexis Midon
A Connection and a ZK session are two different things. The ZK master keeps track of a client session validity. When a client connection gets interrupted, its associated session goes into Disconnected state, after a while it will be Expired, but if a new connection is established before the time

Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-17 Thread Christian Posta
I believe so. Happy to be corrected. On Wed, Feb 17, 2016 at 12:31 PM, Joe San wrote: > So if I use the High Level Consumer API, using the ConsumerConnector, I get > this automatic zookeeper connection for free? > > On Wed, Feb 17, 2016 at 8:25 PM, Christian Posta < > christian.po...@gmail.com>

Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-17 Thread Joe San
So if I use the High Level Consumer API, using the ConsumerConnector, I get this automatic zookeeper connection for free? On Wed, Feb 17, 2016 at 8:25 PM, Christian Posta wrote: > i believe reconnect is handled automatically by the client... is that what > you're asking > peek here to see how it

Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-17 Thread Christian Posta
i believe reconnect is handled automatically by the client... is that what you're asking peek here to see how it does that and when: https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1153 On Wed, Feb 17, 2016 at 12:14 PM, Joe San wrote: > It is a

Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-17 Thread Joe San
It is all pretty strange. Here is what I see in my logs as soon as I voluntarily shutdown Zookeeper! java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_60] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[

Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-17 Thread Christian Posta
Yep, assuming you haven't completely partitioned that client from the cluster, ZK should automatically try to connect/reconnect to other peers in the server list. Otherwise, it's as Alexis said -- your session would expire; you'd have to recreate the session once you have connectivity On Wed, Feb

Re: Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-17 Thread Alexis Midon
By "re-connect", I'm assuming that the ZK session is expired, not disconnected. For details see http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#ch_zkSessions In that case, the high level consumer is basically dead, and the application should create a new instance of it. On Mon, F

Enable Kafka Consumer 0.8.2.1 Reconnect to Zookeeper

2016-02-15 Thread Joe San
Any ideas as to which property should I set to enable Zookeeper re-connection? I have the following properties defined for my consumer (High Level Consumer API). Is this enough for a automatic Zookeeper re-connect? val props = new Properties() props.put("zookeeper.connect", zookeeper) props.put("g