Re: new producer api and batched Futures....

2014-11-20 Thread Jason Rosenberg
I guess it would make the api less clean, but I can imagine a sendBatch method, which returns a single Future that gets triggered only when all messages in the batch were finished. The callback info could then contain info about the success/exceptions encountered by each sub-group of messages. An

Re: new producer api and batched Futures....

2014-11-20 Thread Jay Kreps
Internally it works as you describe, there is only one CountDownLatch per batch sent, each of the futures is just a wrapper around that. It is true that if you accumulate thousands of futures in a list that may be a fair number of objects you are retaining, and there will be some work involved in