Hello, I have submitted a pull request (Adding support of initial value for state update. #2665), please review and let me know.
Excited to submit my first pull request. -Soumitra. ----- Original Message ----- From: "Soumitra Kumar" <kumar.soumi...@gmail.com> To: dev@spark.apache.org Sent: Tuesday, September 23, 2014 1:28:21 PM Subject: SPARK-3660 : Initial RDD for updateStateByKey transformation Hello fellow developers, Thanks TD for relevant pointers. I have created an issue : https://issues.apache.org/jira/browse/SPARK-3660 Copying the description from JIRA: " How to initialize state tranformation updateStateByKey? I have word counts from previous spark-submit run, and want to load that in next spark-submit job to start counting over that. One proposal is to add following argument to updateStateByKey methods. initial : Option [RDD [(K, S)]] = None This will maintain the backward compatibility as well. I have a working code as well. This thread started on spark-user list at: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-initialize-updateStateByKey-operation-td14772.html " Please let me know if I shall add a parameter "initial : Option [RDD [(K, S)]] = None" to all updateStateByKey methods or create new ones? Thanks, -Soumitra. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
