Re: unpersist RDD from another thread

2015-09-16 Thread Tathagata Das
Yes. On Wed, Sep 16, 2015 at 1:12 PM, Paul Weiss wrote: > So in order to not incur any performance issues I should really wait for > all usage of the rdd to complete before calling unpersist, correct? > > On Wed, Sep 16, 2015 at 4:08 PM, Tathagata Das < > tathagata.das1...@gmail.com> wrote: > >>

Re: unpersist RDD from another thread

2015-09-16 Thread Paul Weiss
So in order to not incur any performance issues I should really wait for all usage of the rdd to complete before calling unpersist, correct? On Wed, Sep 16, 2015 at 4:08 PM, Tathagata Das wrote: > unpredictable. I think it will be safe (as in nothing should fail), but > the performance will be u

Re: unpersist RDD from another thread

2015-09-16 Thread Tathagata Das
unpredictable. I think it will be safe (as in nothing should fail), but the performance will be unpredictable (some partition may use cache, some may not be able to use the cache). On Wed, Sep 16, 2015 at 1:06 PM, Paul Weiss wrote: > Hi, > > What is the behavior when calling rdd.unpersist() from