Hi,
You may want to evaluate Spark JobServer or Livy or something that
would keep your SparkContext alive.
Pozdrawiam,
Jacek Laskowski
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski
On Thu, Jul
I am writing a Spark application that has many iterations.
I am planning to checkpoint on every Nth iteration to cut the graph of my
rdd and clear previous shuffle files.
I would also like to be able to restart my application completely using the
last checkpoint.
I understand that regular checkpoi
