Re: Broadcast destroy

2017-01-02 Thread Anastasios Zouzias
Hi Bryan, I think the ContextCleaner will take care of the broadcasted variables, see i.e., https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-service-contextcleaner.html If it is easy to spot when to cleanup the broadcast variables in your case, a "xBroadcasted.destroy()" w

RE: Broadcast destroy

2017-01-02 Thread bryan.jeffrey
All, Anyone have a thought? Thank you, Bryan Jeffrey From: bryan.jeff...@gmail.com Sent: Friday, December 30, 2016 1:20 PM To: user Subject: Broadcast destroy All, If we are updating broadcast variables do we need to manually destroy the replaced broadcast, or will they be automatically prun