Hi guys, am wondering how the RDD checkpointing <https://spark.apache.org/docs/latest/streaming-programming-guide.html#RDD Checkpointing> works in Spark Streaming. When I use updateStateByKey, does the Spark store the entire state (at one time point) into the HDFS or only put the transformation into the HDFS? Thank you.
Best, Fang, Yan yanfang...@gmail.com +1 (206) 849-4108
