Guozhang, Yes - you identified the problem!
We had inserted the .get() for debugging, but didn’t think of the (huge!) side-effects. Using the async callback works perfectly well. We are now able to send 100,000 records in 14 sec from a laptop to the Bluemix cloud - ~1000x faster, Thank you very much! Gary > On Dec 13, 2015, at 2:48 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > Gary, > > You are calling "kafkaProducer.send(record).get();" for each message, the > get() call block until the Future is initialized, which effectively > synchronize all message sent by asking for the ACK for each message before > sending the next message, hence no batching. > > You can try using "send(record, callback)" for async sending and let the > callback handle errors from the returned metadata. > > Guozhang > > <snipped>
