I don't think this is a bug. Currently we don't support timing out connections in the clients, which would be a good feature to add. As a result the connection remains until the LB kills it. When that happens you get a message logged that the connection was unexpectedly closed, which I think is what should happen (you can disable the logging in log4j if you don't want it).
It would be nice to implement a client-side connection LRU for unused connections. I filed a ticket to track this: https://issues.apache.org/jira/browse/KAFKA-1941 -Jay On Tue, Feb 10, 2015 at 11:33 AM, Bhavesh Mistry <mistry.p.bhav...@gmail.com > wrote: > HI Ewen, > > The root of the problem is leak of TCP connection which idle for while. It > is just a log message as you mentioned, but suppose you have 50 or more > producer instances created by application and everyone of then will print > above log that becomes little concern. > > We configured producer with bootstrap list as "LB:port1" and it is set to > TCP port forward to "broker:port2". When producer fetches Cluster Metadata > and discovers that TCP connection "LB:port1" is not part of "broker > cluster or topology", it should close connection to "LB:port1" (In my > opinion, this would be expected behavior). > > As you mentioned, producer behavior is normal and this error is harmless. > If you consider this as a bug, please let me know and I will file jira > ticket for this. > > We are on non-release 0.8.2 from trunk. > > Thanks, > > Bhavesh > > > > Thanks, > > Bhavesh > > On Tue, Feb 10, 2015 at 12:29 AM, Ewen Cheslack-Postava <e...@confluent.io > > > wrote: > > > Bhavesh, > > > > I'm unclear what the impact is here. The line numbers don't match up > > exactly with trunk or 0.8.2.0, but it looks like this exception is just > > caught and logged. As far as I can tell the producer would continue to > > function normally. Does this have any impact on the producer or is the > > concern just that the exception is being logged? > > > > On Mon, Feb 9, 2015 at 11:21 PM, Bhavesh Mistry < > > mistry.p.bhav...@gmail.com> > > wrote: > > > > > HI Kafka Team, > > > > > > Please confirm if you would like to open Jira issue to track this ? > > > > > > Thanks, > > > > > > Bhavesh > > > > > > On Mon, Feb 9, 2015 at 12:39 PM, Bhavesh Mistry < > > > mistry.p.bhav...@gmail.com> > > > wrote: > > > > > > > Hi Kakfa Team, > > > > > > > > We are getting this connection reset by pears after couple of minute > > > aster > > > > start-up of producer due to infrastructure deployment strategies we > > have > > > > adopted from LinkedIn. > > > > > > > > We have LB hostname and port as seed server, and all producers are > > > getting > > > > following exception because of TCP idle connection timeout set on LB > > > (which > > > > is 2 minutes and Kafka TCP connection idle is set to 10 minutes). > > This > > > > seems to be minor bug to close TCP connection after discovering that > > > seed > > > > server is not part of brokers list immediately. > > > > > > > > > > > > java.io.IOException: Connection reset by peer > > > > at sun.nio.ch.FileDispatcher.read0(Native Method) > > > > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) > > > > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198) > > > > at sun.nio.ch.IOUtil.read(IOUtil.java:171) > > > > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245) > > > > at > > > > > > > > > > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:60) > > > > at > org.apache.kafka.common.network.Selector.poll(Selector.java:242) > > > > at > > > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:178) > > > > at > > > > > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:175) > > > > at > > > > > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:115) > > > > at java.lang.Thread.run(Thread.java:662) > > > > java.io.IOException: Connection reset by peer > > > > at sun.nio.ch.FileDispatcher.read0(Native Method) > > > > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) > > > > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198) > > > > at sun.nio.ch.IOUtil.read(IOUtil.java:171) > > > > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245) > > > > at > > > > > > > > > > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:60) > > > > at > org.apache.kafka.common.network.Selector.poll(Selector.java:242) > > > > at > > > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:178) > > > > at > > > > > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:175) > > > > at > > > > > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:115) > > > > at java.lang.Thread.run(Thread.java:662) > > > > > > > > > > > > Thanks, > > > > > > > > Bhavesh > > > > > > > > > > > > > > > > > > > -- > > Thanks, > > Ewen > > >
