Also keep in mind that unfortunately KafkaConsumer.poll(..) will deadlock regardless of the timeout if connection to the broker can not be established and won't react to thread interrupts. This essentially means that the only way to exit is to kill jvm. This is all because Kafka fetches topic metadata synchronously before timeout takes effect. While it is my understanding that the reason for it is there is a background thread attempting to reconnect in the event of temporary broker outage, it doesn't help if you accidentally specified wrong broker url.
Oleg > On Aug 2, 2016, at 10:27, Kamal C <kamaltar...@gmail.com> wrote: > > See the answers inline. > >> On Tue, Aug 2, 2016 at 12:23 AM, sat <sathish.al...@gmail.com> wrote: >> >> Hi, >> >> I am new to Kafka. We are planning to use Kafka messaging for our >> application. I was playing with Kafka 0.9.0.1 version and i have following >> queries. Sorry for asking basic questions. >> >> >> 1) I have instantiated Kafka Consumer and invoked >> consumer.poll(Long.MAX_VALUE). Although i have specified timeout as >> Long.MAX_VALUE, i observe my consumer to fetch records whenever the >> publisher publishes a message to a topic. This makes me wonder whether >> Kafka Consumer is push or pull mechanism. Please help us understand the >> logic of consumer.poll(timeout). > > Fetches the data from the topic, waiting up to the specified wait time *if > necessary *for a record to become available. > Kafka Consumer by design is pull mechanism. > > Take a look into Kafka Consumer java docs[1]. It's explained in detail. > > >> 2) What are the pros and cons of poll for long timeout vs short timeout. >> >> Short Timeout > > Pros: > - On shutdown, if no data available in the topic -- Shutdown will be quick > > Cons: > - Number of network trips will be high > > >> >> Thanks and Regards >> A.SathishKumar > > [1]: > https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html > > -- Kamal
