Thanks Jun. I grep the server log, and found 3 kinds of exception on broker.
1. kafka.common.OffsetOutOfRangeException I don't think this could cause broker closing the socket. 2. 579740110 [kafka-processor-6] ERROR kafka.network.Processor - Closing socket for /10.47.7.56 because of error java.io.IOException: Broken pipe at sun.nio.ch.FileChannelImpl.transferTo0(Native Method) at sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:405) at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:506) at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102) at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53) at kafka.network.MultiSend.writeTo(Transmission.scala:91) at kafka.network.Processor.write(SocketServer.scala:339) at kafka.network.Processor.run(SocketServer.scala:216) at java.lang.Thread.run(Thread.java:662) 3. 580861724 [kafka-processor-13] ERROR kafka.network.Processor - Closing socket for /10.47.7.56 because of error java.nio.channels.ClosedChannelException at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88) at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:486) at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102) at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53) at kafka.network.MultiSend.writeTo(Transmission.scala:91) at kafka.network.Processor.write(SocketServer.scala:339) at kafka.network.Processor.run(SocketServer.scala:216) at java.lang.Thread.run(Thread.java:662) probably the second exception may be the cause. also, I have found the EOFException on consumer side doesn't cause stopping fetching every time. On Fri, May 3, 2013 at 11:50 PM, Jun Rao <jun...@gmail.com> wrote: > The consumer hit the exception because the broker closed the socket. What > does the broker log around the same time say? It should tell you the reason > why the broker closed the socket. > > Thanks, > > Jun > > > On Fri, May 3, 2013 at 2:34 AM, Helin Xiang <xkee...@gmail.com> wrote: > > > hi, > > > > We are doing some mirror-maker test。We currently use version 0.7.2。 > > > > some times,the mirror maker will throw exceptions like this, and then > > stopped fetching topics: > > > > java.io.EOFException: Received -1 when reading from channel, socket has > > likely been closed. > > at kafka.utils.Utils$.read(Utils.scala:539) > > at > > > > > kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:67) > > at kafka.network.Receive$class.readCompletely(Transmission.scala:55) > > at > > > > > kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29) > > at > kafka.consumer.SimpleConsumer.getResponse(SimpleConsumer.scala:177) > > at > > kafka.consumer.SimpleConsumer.liftedTree2$1(SimpleConsumer.scala:125) > > at > kafka.consumer.SimpleConsumer.multifetch(SimpleConsumer.scala:115) > > at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:60) > > [2013-05-02 15:13:05,339] INFO stopping fetcher FetchRunnable-0 to host > > kf.a.com (kafka.consumer.FetcherRunnable) > > > > I used jconsole to double check, and confirmed that the consumer in the > > mirror maker stopped fetching logs brokers。I have to stop the mirror > maker > > and start it over, and it continued fetching logs。 > > > > I start consumers using only 1 consumer thread。 > > > > By the way,there are 16 mirror makers in the consume group。 > > I did some calculation and it can be assured that the partition numbers > are > > larger than or equal to the consumer threads。 > > > > Thats all I got。So, I wonder if there was a bug issue for just mirror > maker > > or any applications using the high level consumers will behave like this? > > > > > > thanks! > > > > > > -- > > *Best Regards > > > > Helin Xiang* > > > -- *Best Regards 向河林*