Any error in the broker, controller and state change log? Thanks,
Jun On Tue, Apr 8, 2014 at 3:08 PM, Sotaro Kimura <rfbrin...@gmail.com> wrote: > Hi, all. > > Sorry for my late reply. > > I tried advertised.host.name/advertised.port. > But producer failed fetching metadata from kafka broker. > > Configuration is below. > - Kafka cluster in DataCenter(each machine has Grobal IP Address). > Firewall allows only ssh port(22). > ex) 192.168.100.100 > - Kafka producer exists out of DataCenter. > ex) 172.16.0.100 > > Broker additional setting is below. > advertised.host.name=localhost > advertised.port=19092 > > Kafka producer operation is below. > > - SSH port fowarding > ======================== > # ssh root@192.168.100.100 -L 19092:192.168.100.100:9092 -L 12181: > 192.168.100.100:2181 > // Kafka broker and zookeeper ssh port fowarding. > ======================== > > - Producer data put(failed) > ======================== > # ./kafka-topics.sh --create --zookeeper localhost:12181 > --replication-factor 1 --partitions 3 --topic TestTopic > Created topic "TestTopic". > // Succeed create topic over ssh port fowarding. > # ./kafka-topics.sh --list --zookeeper localhost:12181 > TestTopic > // Succeed get topic list over ssh port fowarding. > # ./kafka-console-producer.sh --broker-list localhost:19092 --topic > TestTopic > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > SLF4J: Defaulting to no-operation (NOP) logger implementation > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further > details. > Test > [2014-04-06 20:42:01,352] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,364] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,365] ERROR Failed to collate messages by topic, > partition due to: Failed to fetch topic metadata for topic: TestTopic > (kafka.producer.async.DefaultEventHandler) > [2014-04-06 20:42:01,476] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,481] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,481] ERROR Failed to collate messages by topic, > partition due to: Failed to fetch topic metadata for topic: TestTopic > (kafka.producer.async.DefaultEventHandler) > [2014-04-06 20:42:01,589] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,595] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,595] ERROR Failed to collate messages by topic, > partition due to: Failed to fetch topic metadata for topic: TestTopic > (kafka.producer.async.DefaultEventHandler) > [2014-04-06 20:42:01,709] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartit > ionInfo) > [2014-04-06 20:42:01,719] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,719] ERROR Failed to collate messages by topic, > partition due to: Failed to fetch topic metadata for topic: TestTopic > (kafka.producer.async.DefaultEventHandler) > [2014-04-06 20:42:01,826] WARN Error while fetching metadata > [{TopicMetadata for topic TestTopic ->No partition metadata for topic > TestTopic due to kafka.common.LeaderNotAvailableException}] for topic > [TestTopic]: class kafka.common.LeaderNotAvailableException > (kafka.producer.BrokerPartitionInfo) > [2014-04-06 20:42:01,829] ERROR Failed to send requests for topics > TestTopic with correlation ids in [0,8] > (kafka.producer.async.DefaultEventHandler) > [2014-04-06 20:42:01,830] ERROR Error in handling batch of 2 events > (kafka.producer.async.ProducerSendThread) > kafka.common.FailedToSendMessageException: Failed to send messages after 3 > tries. > at > > kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90) > at > > kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104) > at > > kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87) > at > > kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67) > at scala.collection.immutable.Stream.foreach(Stream.scala:526) > at > > kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66) > at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44) > ======================== > > > If advertised setting is not defined, > producer connection is refused.(below error) > So I think advertised setting is working. > > ======================== > # ./kafka-console-producer.sh --broker-list localhost:19092 --topic > TestTopic > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > SLF4J: Defaulting to no-operation (NOP) logger implementation > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further > details. > Test > [2014-04-08 21:11:44,231] ERROR Producer connection to localhost:9092 > unsuccessful (kafka.producer.SyncProducer) > java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > ======================== > > Is there anyone you find the cause of error. > > regards. > > > > > 2014-03-20 8:00 GMT+09:00 Sotaro Kimura <rfbrin...@gmail.com>: > > > Thanks for quick reply. > > > > > > advertised.host.name/advertised.port setting is this. > > I confirmed only producer setting, so I couldn't find these properties. > > > > I will try it. Thank you. > > > > > > 2014-03-20 7:50 GMT+09:00 Joe Stein <joe.st...@stealth.ly>: > > > > With 0.8.1 update these two properties in the server.properties config of > >> the broker to be 172.16.0.100 and 19092 respectively > >> > >> # Hostname the broker will advertise to producers and consumers. If not > >> set, it uses the > >> # value for "host.name" if configured. Otherwise, it will use the > value > >> returned from > >> # java.net.InetAddress.getCanonicalHostName(). > >> #advertised.host.name=<hostname routable by clients> > >> > >> # The port to publish to ZooKeeper for clients to use. If this is not > set, > >> # it will publish the same port that the broker binds to. > >> #advertised.port=<port accessible by clients> > >> > >> /******************************************* > >> Joe Stein > >> Founder, Principal Consultant > >> Big Data Open Source Security LLC > >> http://www.stealth.ly > >> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >> ********************************************/ > >> > >> > >> On Wed, Mar 19, 2014 at 6:39 PM, Sotaro Kimura <rfbrin...@gmail.com> > >> wrote: > >> > >> > Hi, all. > >> > > >> > I'm trying Log produce to Remote Kafka Cluster over SSH Tunneling. > >> > > >> > Configuration is below. > >> > - Kafka cluster in DataCenter(each machine has Grobal IP Address). > >> > Firewall allows only ssh port(22). > >> > ex) 192.168.100.100 > >> > - Kafka producer exists out of DataCenter. > >> > ex) 172.16.0.100 > >> > > >> > I tried produce over SSH Tunneling. > >> > Producer successed getting meta data from brokers. > >> > But meta data is Address which producer can't access directry. > >> > > >> > Detail is below. > >> > 1. Ssh connect producer machine to DataCenter. > >> > 172.16.0.100 > 192.168.100.100:22 > >> > 2. Create SSH Tunnel over ssh connection. > >> > 172.16.0.100's localhost:19092 to 192.168.100.100:9092(Kafka receive > >> > port). > >> > 3. Producer connect to localhost:19092. > >> > Producer get metadata from brokers(192.168.100.100:9092). > >> > 4. Producer failed to connect brokers(192.168.100.100:9092). > >> > > >> > In such situations, how I can produce to Remote Kafka Cluster? > >> > > >> > -- > >> > ################################# > >> > 木村 宗太郎 Sotaro Kimura > >> > <rfbrin...@gmail.com> > >> > ################################# > >> > > >> > > > > > > > > -- > > ################################# > > 木村 宗太郎 Sotaro Kimura > > <rfbrin...@gmail.com> > > ################################# > > > > > > -- > ################################# > 木村 宗太郎 Sotaro Kimura > <rfbrin...@gmail.com> > ################################# >
