The error you saw on the broker is for consumer requests, not for producer. For the issues in the producer, are you using a VIP? Is there any firewall btw producer and broker? The typical issues with "connection reset" that we have seen are caused by the load balancer or the firewall killing idle connections.
Thanks, Jun On Sun, Mar 17, 2013 at 8:24 PM, Yonghui Zhao <zhaoyong...@gmail.com> wrote: > In kafka 0.7.2, I use a producer to send 200 million message to kafka > server. > > After sent 100 million this exception happend: > > In producer: > > Exception in thread "main" java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcher.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:182) > at sun.nio.ch.SocketChannelImpl.write0(SocketChannelImpl.java:383) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:406) > at java.nio.channels.SocketChannel.write(SocketChannel.java:384) > at > kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:49) > at kafka.network.Send$class.writeCompletely(Transmission.scala:73) > at > > kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25) > at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:95) > at kafka.producer.SyncProducer.send(SyncProducer.scala:94) > at kafka.producer.SyncProducer.send(SyncProducer.scala:125) > at > > kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:114) > at > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100) > at > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100) > at > > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43) > at kafka.producer.ProducerPool.send(ProducerPool.scala:100) > at kafka.producer.Producer.zkSend(Producer.scala:137) > at kafka.producer.Producer.send(Producer.scala:99) > at kafka.javaapi.producer.Producer.send(Producer.scala:103) > > > In kafka server: > > [2013-03-16 06:59:49,491] ERROR Closing socket for /10.2.201.201 because > of > error (kafka.network.Processor) > java.io.IOException: Connection reset by peer > at sun.nio.ch.FileChannelImpl.transferTo0(Native Method) > at > sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:456) > at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:557) > at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102) > at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53) > at kafka.network.MultiSend.writeTo(Transmission.scala:91) > at kafka.network.Processor.write(SocketServer.scala:339) > at kafka.network.Processor.run(SocketServer.scala:216) > at java.lang.Thread.run(Thread.java:679) > > Have you ever seen this exception before, what's the root cause? Thanks >
