RE: 0.9.0.1 Kafka Java client Producer hangs while requesting metadata

[Muthukumaran K]

With metadata.fetch.timeout.ms=1, Producer sending goes through. Not sure if 
this is going to affect anything else. 
Now trying to figure-out why KafkaConsumer.poll(10000) never returns. 

Regards
Muthu


-----Original Message-----
From: Muthukumaran K [mailto:muthukumara...@ericsson.com] 
Sent: Wednesday, March 02, 2016 11:28 PM
To: users@kafka.apache.org
Subject: 0.9.0.1 Kafka Java client Producer hangs while requesting metadata

Hi,

Trying a very simple Producer with following code. "producer.send" hangs 
indefinitely. Attaching thread-dump snippet so that I can get some advice if 
something is wrong with my code or configuration.

Kafka runs in a VM and producer runs in host - it's a single-broker setup for 
basic testing. Kafka version used is is 0.9.0.1


Should I set - metadata.fetch.timeout.ms=1 to get over this ? Any side-effects ?



    public static void main(String args[]) throws InterruptedException, 
ExecutionException {


        Properties props = new Properties();
        props.put("bootstrap.servers", 
"<REMOTE-IP-REACHABLE-FROM-LOCAL-MC>:9092");
        //props.put("acks", "all");
        props.put("retries", 0);
        props.put("batch.size", 16384);
        props.put("linger.ms<http://linger.ms/>", 1);
        props.put("buffer.memory", 33554432);
        props.put("key.serializer", 
"org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", 
"org.apache.kafka.common.serialization.StringSerializer");

        Producer<String, String> producer = new KafkaProducer<>(props);

        System.out.println("Connected ...");

        for(int i = 0; i < 100; i++){
            producer.send(new ProducerRecord<String, 
String>("consumer-tutorial", Integer.toString(i), Integer.toString(i)));
            System.out.println("Sent " + i);
        }

        producer.close();
    }


Thread Dump snippet :

"main" prio=6 tid=0x0000000002238000 nid=0x1390 in Object.wait() 
[0x00000000025bf000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
            at java.lang.Object.wait(Native Method)
            - waiting on <0x00000007aecacea0> (a 
org.apache.kafka.clients.Metadata)
            at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:121)
            - locked <0x00000007aecacea0> (a org.apache.kafka.clients.Metadata)
            at 
org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:483)
            at 
org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:412)
            at 
org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:339)
            at kafkaexperiments.ProducerDriver.main(ProducerDriver.java:41)

Additional checks and info on my setup

1)  bin/kafka-topics.sh --list --zookeeper <VM-IP>:2181  - this clearly shows 
that the topic "consumer-tutorial" is present
2)  to check if there are issues with kafka itself, I locally (within VM where 
Kafka and ZK runs) checked with kafka-console-producer.sh and 
kafka-console-consumer.sh with a different topic and could verify that there is 
no issue with Kafka setup. It worked fine
3) I am using following imports

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerRecord;



Regards
Muthu