Re: Out of scope RDDs not getting cleaned up

Log file mentioned in the post --> FuturesTImeoutException.txt -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ ---

Out of scope RDDs not getting cleaned up

Hi, I am using spark 2.1 and I am leveraging spark streaming for my data pipeline. Now, in my case the batch size is 3 minutes and we persist couple of RDDs while processing a batch and after processing we rely on Spark's ContextCleaner to clean out RDDs which are no longer in scope. So we have s