The consumer uses a sync.producer to look up topic metadata for topics that it consumes. For some reason, it is unable to establish a connection to the brokers. Are the brokers all up with clean logs?
On Mon, Aug 12, 2013 at 8:39 AM, Eric Sites <eric.si...@threattrack.com> wrote: > I am getting some new messages in my 0.8 Java producer app. > > I am running 6 Kafka brokers. > > The app producing these logs messages is not doing anything, i.e. Not > producing messages. > > Anywhere form every 5 minutes to 10 seconds sometimes. > > [2013-08-12 14:45:20,714] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > java.nio.channels.ClosedByInterruptException > at > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.producer.SyncProducer.connect(SyncProducer.scala:146) > at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161) > at > kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68) > at kafka.producer.SyncProducer.send(SyncProducer.scala:112) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88) > at > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:64) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > [2013-08-12 14:45:20,714] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > java.nio.channels.ClosedByInterruptException > at > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.producer.SyncProducer.connect(SyncProducer.scala:146) > at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161) > at > kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68) > at kafka.producer.SyncProducer.send(SyncProducer.scala:112) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88) > at > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:64) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > [2013-08-12 14:45:20,715] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > java.nio.channels.ClosedByInterruptException > at > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.producer.SyncProducer.connect(SyncProducer.scala:146) > at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161) > at > kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68) > at kafka.producer.SyncProducer.send(SyncProducer.scala:112) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88) > at > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:64) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > [2013-08-12 14:45:20,715] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > java.nio.channels.ClosedByInterruptException > at > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.producer.SyncProducer.connect(SyncProducer.scala:146) > at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161) > at > kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68) > at kafka.producer.SyncProducer.send(SyncProducer.scala:112) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88) > at > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:64) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > [2013-08-12 14:45:20,716] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > java.nio.channels.ClosedByInterruptException > at > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.producer.SyncProducer.connect(SyncProducer.scala:146) > at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161) > at > kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68) > at kafka.producer.SyncProducer.send(SyncProducer.scala:112) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53) > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88) > at > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:64) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > > > [root@iris001 iris]# cat kafka_iris_server.log | grep kafka00 > [2013-08-12 14:35:39,261] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:35:39,263] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:35:39,264] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:35:39,264] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:35:39,264] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:25,504] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:25,504] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:25,505] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:25,505] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:25,505] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:57,131] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:57,132] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:57,132] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:57,132] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:37:57,132] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:40:14,811] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:40:14,812] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:40:14,812] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:40:14,812] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:40:14,812] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:45:20,714] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:45:20,714] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:45:20,715] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:45:20,715] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 14:45:20,716] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:01:02,629] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:01:02,629] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:01:02,630] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:01:02,630] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:01:02,631] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:02:26,764] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:02:26,764] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:02:26,765] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:02:26,765] ERROR Producer connection to kafka003:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:02:26,765] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:08:15,401] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:08:15,405] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:08:15,409] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:08:15,410] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:08:15,410] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:11:57,408] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:11:57,409] ERROR Producer connection to kafka003:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:11:57,410] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:11:57,410] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:11:57,410] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:13:35,083] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:13:35,084] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:13:35,084] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:13:35,084] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:13:35,085] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:20:07,381] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:20:07,382] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:20:07,382] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:20:07,382] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:20:07,383] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:23:59,516] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:23:59,517] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:23:59,518] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:23:59,518] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:23:59,518] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:28:23,664] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:28:23,665] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:28:23,665] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:28:23,665] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:28:23,666] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:31:01,415] ERROR Producer connection to kafka005:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:31:01,417] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:31:01,417] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:31:01,417] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:31:01,418] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:36:18,623] ERROR Producer connection to kafka004:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:36:18,623] ERROR Producer connection to kafka001:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:36:18,624] ERROR Producer connection to kafka003:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:36:18,624] ERROR Producer connection to kafka002:9092 > unsuccessful (kafka.producer.SyncProducer) > [2013-08-12 15:36:18,625] ERROR Producer connection to kafka006:9092 > unsuccessful (kafka.producer.SyncProducer) > > Thanks for the help... > > Cheers, > Eric Sites
