in ALS， I guess all the iteration’s rdds are referenced by its next
iteration’s rdd, so all the shuffle data will not be deleted until the als job
finished…
I guess checkpoint could solve my problem, do you know checkpoint?
> 在 2015年3月3日，下午4:18，nitin [via Apache Spark User List]
> 写道：
>
> S
Shuffle write will be cleaned if it is not referenced by any object
directly/indirectly. There is a garbage collector written inside spark which
periodically checks for weak references to RDDs/shuffle write/broadcast and
deletes them.
--
View this message in context:
http://apache-spark-user-li
Can't find anything related to this from the Configurations page
http://spark.apache.org/docs/1.2.0/configuration.html, You could probably
open a JIRA issue regarding this.
Thanks
Best Regards
On Tue, Mar 3, 2015 at 12:03 PM, lisendong wrote:
> I 'm using spark als.
>
> I set the iteration numb
