Hey Alexey, looks like we didn't have a JIRA before, but I created one just now: https://issues.apache.org/jira/browse/KAFKA-3412. As soon as I get a little time, I'll polish up my old patch and submit it.
-Jason On Wed, Mar 16, 2016 at 4:10 AM, Alexey Romanchuk < alexey.romanc...@gmail.com> wrote: > Jason, Jay, could you please share bug number to track this issue? > > On Tue, Mar 15, 2016 at 10:57 PM, Jason Gustafson <ja...@confluent.io> > wrote: > > > Yeah, I agree the bug is probably more serious than I had thought before > > (I've gotten too used to examples with only a single commitAsync() > call). I > > had worked on a patch to fix this at one point. Let me see if I can dig > it > > up. > > > > -Jason > > > > On Tue, Mar 15, 2016 at 1:06 AM, Alexey Romanchuk < > > alexey.romanc...@gmail.com> wrote: > > > > > Jay, I am completely agreed with you. From API level it is nothing I > can > > do > > > about it. Anything else I have to do with this issue? Reproducible > > example > > > here - https://gist.github.com/13h3r/42633bcd64b80ddffe6b > > > > > > Thanks! > > > > > > On Tue, Mar 15, 2016 at 11:24 AM, Jay Kreps <j...@confluent.io> wrote: > > > > > > > This seems like a bug, no? It should just initiate the request not > wait > > > for > > > > it to be written, there is no way for the user to reason about the > > state > > > of > > > > the send buffer. > > > > > > > > -jay > > > > > > > > On Monday, March 14, 2016, Jason Gustafson <ja...@confluent.io> > wrote: > > > > > > > > > Hey Alexey, > > > > > > > > > > Asynchronous commit handling could probably be improved quite a > bit. > > > > > Basically what's happening is that the client's send buffer is > > getting > > > > > filled up, which causes the subsequent commits to fail with > > > > > SendFailedException. We don't currently implement any retrying for > > > > > asynchronous commits, so even transient failures get bubbled up to > > the > > > > > user. This is.. unfortunate. We probably can implement some retry > > > logic, > > > > > but we need to be careful careful about preserving the commit > order. > > > > Maybe > > > > > this is as simple as adding sequence numbers for each partition. I > > > > haven't > > > > > thought it all the way through, but you're welcome to open a JIRA > if > > > you > > > > > think it would be helpful. > > > > > > > > > > For now, the easiest way to mitigate the problem is probably to > batch > > > the > > > > > commits into a single call to commitAsync(). Would that work for > you? > > > > > > > > > > -Jason > > > > > > > > > > On Mon, Mar 14, 2016 at 8:18 AM, Alexey Romanchuk < > > > > > alexey.romanc...@gmail.com <javascript:;>> wrote: > > > > > > > > > > > Hi all! > > > > > > > > > > > > I am using new client 0.9.0.1. > > > > > > > > > > > > I found that when I call commitAsync multiple times before > calling > > > poll > > > > > > most of commits failed with SendFailedException. > > > > > > > > > > > > Here it is an example of code - > > > > > > https://gist.github.com/13h3r/42633bcd64b80ddffe6b > > > > > > > > > > > > Could you please explain commitAsync in more details? > > > > > > > > > > > > What is wrong with calling commitAsync multiple times? Should I > > call > > > > poll > > > > > > immediate after calling commitAsync? Should I precalculate > offsets > > > for > > > > > > multiple commits for every topic/partition to minimize number of > > > > offsets? > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > >
