Thanks a lot for the response. On Tue, Sep 18, 2018 at 3:21 PM, Matthias J. Sax <matth...@confluent.io> wrote:
> You will need to `poll()` -- that is how Kafka works. > > The brokers practically don't even know that a consumer exist and thus > cannot sent any notification. It's be design. > > > -Matthias > > On 9/18/18 8:26 AM, Siddhartha Khaitan wrote: > > Hello, > > > > I have a basic question. > > > > Lets say a kafka consumer is subscribed to to a topic "test". When the > > producer publishes logs/records into that topic, does the consumer > > explicitly need to poll at regular intervals to find out if a new message > > is available and pull the new message or a notification will be sent to > the > > consumer that a new message is available on the topic subscribed and then > > it can call a consume API. Is it possible to do a select on the > connection > > with the broker rather than poll. My kafka cluster will be over WAN and > the > > consumers will be all over geographically. Polling seems to be too > costly. > > > > I am planning on using librdkafka. > > > > I will greatly appreciate any help in this regards, > > > > Regards > > Siddhartha > > > > -- Siddhartha Kumar Khaitan, Ph.D. Research Assistant Professor Electrical and Computer Engineering Iowa State University 1124 Coover Hall 2520 Osborn Dr. Ames IA 50011-1046, USA 5152940906 skhai...@iastate.edu
