[ https://issues.apache.org/jira/browse/KAFKA-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gwen Shapira updated KAFKA-2738: -------------------------------- Description: Scenario (as carefully documented by [~benstopford]: 1. Start 2 or more brokers with listeners on both PLAINTEXT and SSL protocols, and PLAINTEXT as security.inter.broker.protocol: inter.broker.protocol.version = 0.9.0.X security.inter.broker.protocol = PLAINTEXT listeners = PLAINTEXT://:9092,SSL://:9093 2. Stop one of the brokers and change security.inter.broker.protocol to SSL inter.broker.protocol.version = 0.9.0.X security.inter.broker.protocol = SSL listeners = PLAINTEXT://:9092,SSL://:9093 3. Start that broker again. You will get replication errors as it will attempt to use SSL on a PLAINTEXT port: {code} WARN ReplicaFetcherThread-0-3, Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@78ca3ba1. Possible cause: java.io.IOException: Connection to Node(3, worker4, 9092) failed (kafka.server.ReplicaFetcherThread) WARN Failed to send SSL Close message (org.apache.kafka.common.network.SslTransportLayer) java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:492) at org.apache.kafka.common.network.SslTransportLayer.flush(SslTransportLayer.java:188) at org.apache.kafka.common.network.SslTransportLayer.close(SslTransportLayer.java:161) at org.apache.kafka.common.network.KafkaChannel.close(KafkaChannel.java:50) at org.apache.kafka.common.network.Selector.close(Selector.java:448) at org.apache.kafka.common.network.Selector.poll(Selector.java:316) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270) at kafka.utils.NetworkClientBlockingOps$.recurse$1(NetworkClientBlockingOps.scala:128) at kafka.utils.NetworkClientBlockingOps$.kafka$utils$NetworkClientBlockingOps$$pollUntilFound$extension(NetworkClientBlockingOps.scala:139) at kafka.utils.NetworkClientBlockingOps$.kafka$utils$NetworkClientBlockingOps$$pollUntil$extension(NetworkClientBlockingOps.scala:105) at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:58) at kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:202) at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:192) at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:102) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) {code} was: Scenario (as carefully documented by [~benstopford]: 1. Start 2 or more brokers with listeners on both PLAINTEXT and SSL protocols, and PLAINTEXT as security.inter.broker.protocol: inter.broker.protocol.version = 0.9.0.X security.inter.broker.protocol = PLAINTEXT listeners = PLAINTEXT://:9092,SSL://:9093 2. Stop one of the brokers and change security.inter.broker.protocol to SSL inter.broker.protocol.version = 0.9.0.X security.inter.broker.protocol = SSL listeners = PLAINTEXT://:9092,SSL://:9093 3. Start that broker again. You will get replication errors as it will attempt to use SSL on a PLAINTEXT port: <code> WARN ReplicaFetcherThread-0-3, Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@78ca3ba1. Possible cause: java.io.IOException: Connection to Node(3, worker4, 9092) failed (kafka.server.ReplicaFetcherThread) WARN Failed to send SSL Close message (org.apache.kafka.common.network.SslTransportLayer) java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:492) at org.apache.kafka.common.network.SslTransportLayer.flush(SslTransportLayer.java:188) at org.apache.kafka.common.network.SslTransportLayer.close(SslTransportLayer.java:161) at org.apache.kafka.common.network.KafkaChannel.close(KafkaChannel.java:50) at org.apache.kafka.common.network.Selector.close(Selector.java:448) at org.apache.kafka.common.network.Selector.poll(Selector.java:316) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270) at kafka.utils.NetworkClientBlockingOps$.recurse$1(NetworkClientBlockingOps.scala:128) at kafka.utils.NetworkClientBlockingOps$.kafka$utils$NetworkClientBlockingOps$$pollUntilFound$extension(NetworkClientBlockingOps.scala:139) at kafka.utils.NetworkClientBlockingOps$.kafka$utils$NetworkClientBlockingOps$$pollUntil$extension(NetworkClientBlockingOps.scala:105) at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:58) at kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:202) at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:192) at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:102) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) </code> > Can't set SSL as inter-broker-protocol by rolling restart of brokers > -------------------------------------------------------------------- > > Key: KAFKA-2738 > URL: https://issues.apache.org/jira/browse/KAFKA-2738 > Project: Kafka > Issue Type: Bug > Reporter: Gwen Shapira > > Scenario (as carefully documented by [~benstopford]: > 1. Start 2 or more brokers with listeners on both PLAINTEXT and SSL > protocols, and PLAINTEXT as security.inter.broker.protocol: > inter.broker.protocol.version = 0.9.0.X > security.inter.broker.protocol = PLAINTEXT > listeners = PLAINTEXT://:9092,SSL://:9093 > 2. Stop one of the brokers and change security.inter.broker.protocol to SSL > inter.broker.protocol.version = 0.9.0.X > security.inter.broker.protocol = SSL > listeners = PLAINTEXT://:9092,SSL://:9093 > 3. Start that broker again. > You will get replication errors as it will attempt to use SSL on a PLAINTEXT > port: > {code} > WARN ReplicaFetcherThread-0-3, Error in fetch > kafka.server.ReplicaFetcherThread$FetchRequest@78ca3ba1. Possible cause: > java.io.IOException: Connection to Node(3, worker4, 9092) failed > (kafka.server.ReplicaFetcherThread) > WARN Failed to send SSL Close message > (org.apache.kafka.common.network.SslTransportLayer) > java.io.IOException: Broken pipe > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) > at sun.nio.ch.IOUtil.write(IOUtil.java:65) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:492) > at > org.apache.kafka.common.network.SslTransportLayer.flush(SslTransportLayer.java:188) > at > org.apache.kafka.common.network.SslTransportLayer.close(SslTransportLayer.java:161) > at org.apache.kafka.common.network.KafkaChannel.close(KafkaChannel.java:50) > at org.apache.kafka.common.network.Selector.close(Selector.java:448) > at org.apache.kafka.common.network.Selector.poll(Selector.java:316) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270) > at > kafka.utils.NetworkClientBlockingOps$.recurse$1(NetworkClientBlockingOps.scala:128) > at > kafka.utils.NetworkClientBlockingOps$.kafka$utils$NetworkClientBlockingOps$$pollUntilFound$extension(NetworkClientBlockingOps.scala:139) > at > kafka.utils.NetworkClientBlockingOps$.kafka$utils$NetworkClientBlockingOps$$pollUntil$extension(NetworkClientBlockingOps.scala:105) > at > kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:58) > at > kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:202) > at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:192) > at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:102) > at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)