Re: Kafka Consumer Pre Fetch Messages + Async commits

messages in > the event of failure in any case, so the only difference sync commit > would make would be (possibly) slower run time. > > On Sat, Aug 26, 2017 at 1:07 AM, Julia Wistance > wrote: > > Hi Experts, > > > > A question on what could potentially happen with Spa

Kafka Consumer Pre Fetch Messages + Async commits

Hi Experts, A question on what could potentially happen with Spark Streaming 2.2.0 + Kafka. LocationStrategies says that "new Kafka consumer API will pre-fetch messages into buffers.". If we store offsets in Kafka, currently we can only use a async commits. So, 1 - Could it happen that we commit