[
https://issues.apache.org/jira/browse/KAFKA-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137011#comment-14137011
]
nicu marasoiu edited comment on KAFKA-1282 at 9/17/14 9:53 AM:
---------------------------------------------------------------
Hi,
Unfortunately the client used in console-producer is not very robust with
respect to disconnections, as will detail below. Is this the "old" scala
producer, and can we hope for a resilient behaviour that I can test with the
new java producer?
More specifically, the connection is closed from the broker side, but the
producer is unaware of this. The first message after the close is lost (and is
not retried later). The second message sees the broken channel, outputs the
exception below, and reconnects and is succesfully retried, I can see it
consumed.
[2014-09-17 12:44:12,009] WARN Failed to send producer request with correlation
id 15 to broker 0 with data for partitions [topi,0]
(kafka.producer.async.DefaultEventHandler)
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
at sun.nio.ch.IOUtil.write(IOUtil.java:149)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:483)
at java.nio.channels.SocketChannel.write(SocketChannel.java:493)
at
kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:56)
at kafka.network.Send$class.writeCompletely(Transmission.scala:75)
at
kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:26)
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
was (Author: nmarasoi):
Hi,
Unfortunately the client used in console-producer is not robust with respect to
disconnections, as below. Is this the "old" scala producer, and can we hope for
a resilient behaviour that I can test with the new java producer?
[2014-09-17 12:44:12,009] WARN Failed to send producer request with correlation
id 15 to broker 0 with data for partitions [topi,0]
(kafka.producer.async.DefaultEventHandler)
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
at sun.nio.ch.IOUtil.write(IOUtil.java:149)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:483)
at java.nio.channels.SocketChannel.write(SocketChannel.java:493)
at
kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:56)
at kafka.network.Send$class.writeCompletely(Transmission.scala:75)
at
kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:26)
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
> Disconnect idle socket connection in Selector
> ---------------------------------------------
>
> Key: KAFKA-1282
> URL: https://issues.apache.org/jira/browse/KAFKA-1282
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.8.2
> Reporter: Jun Rao
> Assignee: nicu marasoiu
> Labels: newbie++
> Fix For: 0.9.0
>
> Attachments: 1282_the_ultimate,_close_max_one_per_select.patch,
> KAFKA-1282_Disconnect_idle_socket_connection_in_Selector.patch,
> idleDisconnect.patch
>
>
> To reduce # socket connections, it would be useful for the new producer to
> close socket connections that are idle. We can introduce a new producer
> config for the idle time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
