In 0.7, each producer reconnects to the broker periodically. Take a look at reconnect.interval in http://kafka.apache.org/07/configuration.html
Thanks, Jun On Tue, May 21, 2013 at 7:35 AM, Jason Weiss <jason_we...@rapid7.com> wrote: > I've made some good progress on tuning Kafka for our needs, reaching about > 40,000 events per second in our environment. One thing that I noticed in > all of the Kafka server logs is an inordinate amount (I think?) of Closing > socket connection INFO statements. > > I realize they are INFO, so I assume they are expected. Can someone point > me to a resource or provide a couple sentence overview on the socket > strategy here? I'm curious why 6 producer clients, each using 2 threads, > are opening and closing sockets so much. That's an expensive operation, and > surely it affects performance since these connections are closing anywhere > from 2 seconds to 30 seconds (seemingly randomly). > > Thanks! > > Jason > > > > [2013-05-21 14:31:07,726] INFO Closing socket connection to /10.10.1.133. > (kafka.network.Processor) > [2013-05-21 14:31:08,583] INFO Closing socket connection to /10.10.1.132. > (kafka.network.Processor) > [2013-05-21 14:31:08,930] INFO Closing socket connection to /10.10.1.130. > (kafka.network.Processor) > [2013-05-21 14:31:09,599] INFO Closing socket connection to /10.10.1.133. > (kafka.network.Processor) > [2013-05-21 14:31:10,310] INFO Closing socket connection to /10.10.1.67. > (kafka.network.Processor) > [2013-05-21 14:31:10,436] INFO Closing socket connection to /10.10.1.131. > (kafka.network.Processor) > [2013-05-21 14:31:11,165] INFO Closing socket connection to /10.10.1.134. > (kafka.network.Processor) > [2013-05-21 14:31:13,260] INFO Closing socket connection to /10.10.1.67. > (kafka.network.Processor) > [2013-05-21 14:31:13,591] INFO Closing socket connection to /10.10.1.134. > (kafka.network.Processor) > [2013-05-21 14:31:20,988] INFO Closing socket connection to /10.10.1.130. > (kafka.network.Processor) > [2013-05-21 14:31:22,361] INFO Closing socket connection to /10.10.1.132. > (kafka.network.Processor) > [2013-05-21 14:31:22,459] INFO Closing socket connection to /10.10.1.133. > (kafka.network.Processor) > [2013-05-21 14:31:23,915] INFO Closing socket connection to /10.10.1.131. > (kafka.network.Processor) > [2013-05-21 14:31:24,068] INFO Closing socket connection to /10.10.1.132. > (kafka.network.Processor) > [2013-05-21 14:31:24,947] INFO Closing socket connection to /10.10.1.130. > (kafka.network.Processor) > [2013-05-21 14:31:26,168] INFO Closing socket connection to /10.10.1.131. > (kafka.network.Processor) > [2013-05-21 14:31:26,534] INFO Closing socket connection to /10.10.1.133. > (kafka.network.Processor) > [2013-05-21 14:31:26,536] INFO Closing socket connection to /10.10.1.67. > (kafka.network.Processor) > [2013-05-21 14:31:28,251] INFO Closing socket connection to /10.10.1.134. > (kafka.network.Processor) > [2013-05-21 14:31:30,734] INFO Closing socket connection to /10.10.1.67. > (kafka.network.Processor) > [2013-05-21 14:31:31,322] INFO Closing socket connection to /10.10.1.134. > (kafka.network.Processor) > [2013-05-21 14:31:40,833] INFO Closing socket connection to /10.10.1.130. > (kafka.network.Processor) > [2013-05-21 14:31:41,107] INFO Closing socket connection to /10.10.1.133. > (kafka.network.Processor) > [2013-05-21 14:31:43,123] INFO Closing socket connection to /10.10.1.132. > (kafka.network.Processor) > [2013-05-21 14:31:43,169] INFO Closing socket connection to /10.10.1.131. > (kafka.network.Processor) > [2013-05-21 14:31:43,221] INFO Closing socket connection to /10.10.1.132. > (kafka.network.Processor) > [2013-05-21 14:31:45,833] INFO Closing socket connection to /10.10.1.130. > (kafka.network.Processor) > [2013-05-21 14:31:47,143] INFO Closing socket connection to /10.10.1.133. > (kafka.network.Processor) > [2013-05-21 14:31:48,199] INFO Closing socket connection to /10.10.1.131. > (kafka.network.Processor) > > This electronic message contains information which may be confidential or > privileged. The information is intended for the use of the individual or > entity named above. If you are not the intended recipient, be aware that > any disclosure, copying, distribution or use of the contents of this > information is prohibited. If you have received this electronic > transmission in error, please notify us by e-mail at ( > postmas...@rapid7.com) immediately. >
