Hi, On Tue, Feb 24, 2015 at 4:34 AM, Tathagata Das <tathagata.das1...@gmail.com> wrote:
> There are different kinds of checkpointing going on. updateStateByKey > requires RDD checkpointing which can be enabled only by called > sparkContext.setCheckpointDirectory. But that does not enable Spark > Streaming driver checkpoints, which is necessary for recovering from driver > failures. That is enabled only by streamingContext.checkpoint(...) which > internally calls sparkContext.setCheckpointDirectory and also enables other > stuff. > I see, thank you very much! I'm happy to see I will not have to rewrite the entire application :-) Tobias
