Re: Kafka Consumer Pre Fetch Messages + Async commits

2017-08-29 Thread Julia Wistance
Thanks Cody for the reply. My thoughts were that the time is anyways required to write and commit the offsets to any of the external systems - which are all sync. So why not sync commit of Kafka itself to store the offsets. It helps add another dependency on the application side to check if say MyS

Re: Kafka Consumer Pre Fetch Messages + Async commits

2017-08-28 Thread Cody Koeninger
1. No, prefetched message offsets aren't exposed. 2. No, I'm not aware of any plans for sync commit, and I'm not sure that makes sense. You have to be able to deal with repeat messages in the event of failure in any case, so the only difference sync commit would make would be (possibly) slower ru