Re: Failed to remove RDD error

Unfortunately don't have any logs at the moment. I will post them here if they occur again! On Tue, Jun 3, 2014 at 11:04 AM, Tathagata Das wrote: > It was not intended to be experimental as this improves general > performance. We tested the feature since 0.9, and didnt see any problems. > We n

Re: Failed to remove RDD error

It was not intended to be experimental as this improves general performance. We tested the feature since 0.9, and didnt see any problems. We need to investigate the cause of this. Can you give us the logs showing this error so that we can analyze this. TD On Tue, Jun 3, 2014 at 10:08 AM, Michael

Re: Failed to remove RDD error

Thanks Tathagata, Thanks for all your hard work! In the future, is it possible to mark "experimental" features as such on the online documentation? Thanks, Michael On Mon, Jun 2, 2014 at 6:12 PM, Tathagata Das wrote: > Spark.streaming.unpersist was an experimental feature introduced with > S

Re: Failed to remove RDD error

Spark.streaming.unpersist was an experimental feature introduced with Spark 0.9 (but kept disabled), which actively clears off RDDs that are not useful any more. in Spark 1.0 that has been enabled by default. It is possible that this is an unintended side-effect of that. If spark.cleaner.ttl works

Re: Failed to remove RDD error

Hey Mayur, Thanks for the suggestion, I didn't realize that was configurable. I don't think I'm running out of memory, though it does seem like these errors go away when i turn off the spark.streaming.unpersist configuration and use spark.cleaner.ttl instead. Do you know if there are known issue

Re: Failed to remove RDD error

You can increase your akka timeout, should give you some more life.. are you running out of memory by any chance? Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi On Sat, May 31, 2014 at 6:52 AM, Michael Chang wrote: > I'm

Failed to remove RDD error

I'm running a some kafka streaming spark contexts (on 0.9.1), and they seem to be dying after 10 or so minutes with a lot of these errors. I can't really tell what's going on here, except that maybe the driver is unresponsive somehow? Has anyone seen this before? 14/05/31 01:13:30 ERROR BlockMan