Hey Alexey, Asynchronous commit handling could probably be improved quite a bit. Basically what's happening is that the client's send buffer is getting filled up, which causes the subsequent commits to fail with SendFailedException. We don't currently implement any retrying for asynchronous commits, so even transient failures get bubbled up to the user. This is.. unfortunate. We probably can implement some retry logic, but we need to be careful careful about preserving the commit order. Maybe this is as simple as adding sequence numbers for each partition. I haven't thought it all the way through, but you're welcome to open a JIRA if you think it would be helpful.
For now, the easiest way to mitigate the problem is probably to batch the commits into a single call to commitAsync(). Would that work for you? -Jason On Mon, Mar 14, 2016 at 8:18 AM, Alexey Romanchuk < alexey.romanc...@gmail.com> wrote: > Hi all! > > I am using new client 0.9.0.1. > > I found that when I call commitAsync multiple times before calling poll > most of commits failed with SendFailedException. > > Here it is an example of code - > https://gist.github.com/13h3r/42633bcd64b80ddffe6b > > Could you please explain commitAsync in more details? > > What is wrong with calling commitAsync multiple times? Should I call poll > immediate after calling commitAsync? Should I precalculate offsets for > multiple commits for every topic/partition to minimize number of offsets? > > Thanks! >
