Could you explain a bit more what you want to achieve through batching? Better throughput or atomicity?
Thanks, Jun On Thu, Feb 19, 2015 at 4:09 PM, JAmes Atwill <jatw...@linuxstuff.org> wrote: > Hey Jun, > > That's what I've got right now, semaphore before send() and release in the > callback. Am I correct in understanding that there's no way to do any > batching with KafkaProducer itself (other than have a "bulk" message which > would just be a single message with multiple messages for a particular > Node)? > > JAmes > > On Thu, Feb 19, 2015 at 2:50 PM, Jun Rao <j...@confluent.io> wrote: > > > You can register a callback for each message sent. The callback will be > > called when the message is sent successfully or failed. > > > > Thanks, > > > > Jun > > > > On Tue, Feb 17, 2015 at 4:11 PM, JAmes Atwill <jatw...@linuxstuff.org> > > wrote: > > > > > Hi! > > > > > > I'm using the new KafkaProducer in 0.8.2.0. > > > > > > I have thousands of "Nodes" which receive messages. Each message > > > idempotently mutates the state of the Node, so while duplicate messages > > are > > > fine, missed messages are not. > > > > > > I'm writing these messages into a topic with dozens of partitions. > > > > > > Am I correct in believing that I'll have to manually manage having one > > > message "in flight" per "node" at a time? Or is there a mechanism to > say > > > "This message and all messages after it for this partition were > > rejected"? > > > (or something similar) > > > > > > Thanks! > > > > > > JAmes > > > > > >
