Re: Using mapWithState without a checkpoint

2017-01-23 Thread Shixiong(Ryan) Zhu
Even if you don't need the checkpointing data for recovery, "mapWithState" still needs to use "checkpoint" to cut off the RDD lineage. On Mon, Jan 23, 2017 at 12:30 AM, shyla deshpande wrote: > Hello spark users, > > I do have the same question as Daniel. > > I would like to save the state in Ca

Re: Using mapWithState without a checkpoint

2017-01-23 Thread shyla deshpande
Hello spark users, I do have the same question as Daniel. I would like to save the state in Cassandra and on failure recover using the initialState. If some one has already tried this, please share your experience and sample code. Thanks. On Thu, Nov 17, 2016 at 9:45 AM, Daniel Haviv < daniel.h