Non blocking Kafka producer

2016-06-30 Thread Dan Bahir (BLOOMBERG/ 120 PARK)
Hi, I have an application that needs to be low latency writing to Kafka. With the 0.81 producer I set queue.buffering.max.messages to the number of messages I would like to producer to store in memory and queue.enqueue.timeout.ms to 0 to have the producer throw an exception if the server was

Re: Producer Properties

2016-06-28 Thread Dan Bahir (BLOOMBERG/ 120 PARK)
Hi Chris, The new producer returns a future so it works in an async manner hence no need for the producer.type property. 0.8 -> 0.10 batch.num.messages -> batch.size queue.buffering.max.ms -> linger.ms Your assumptions for the serializers look correct. Take a look at the documentation for bot