Re: Issue Storing offset in Kafka for Spark Streaming Application

2017-10-13 Thread Arpan Rajani
of the process, commit the offsets (note that I use the > original stream instance, not `resultStream` > stream.asInstanceOf[CanCommitOffsets].commitAsync(offsetRanges) > } > > I hope this helps, > > kr, Gerard. > > > > > > > > > > > > On Fri,

Issue Storing offset in Kafka for Spark Streaming Application

2017-10-13 Thread Arpan Rajani
Hi all, In our cluster we have Kafka 0.10.1 and Spark 2.1.0. We are trying to store the offsets in Kafka in order to achieve restartability of the streaming application. ( Using checkpoints, I already implemented, we will require to change code in production hence checkpoint won't work) Checking