Re: Garbage collection in Spark interpreter

2016-07-29 Thread Sachin Janani
Thanks Moon. It means that this is the issue with the scala interpreter itself. Regards, -Sachin On Jul 30, 2016 04:48, "moon soo Lee" wrote: > Hi, > > Here's related issue. > https://issues.scala-lang.org/browse/SI-4331 > > Restart interpreter will be the only way to release memory. > It looks

Re: Garbage collection in Spark interpreter

2016-07-29 Thread moon soo Lee
Hi, Here's related issue. https://issues.scala-lang.org/browse/SI-4331 Restart interpreter will be the only way to release memory. It looks like there're some unreleased memory for keeping variable and generated class, even if you're manually releasing reference to the object, like var myObject

Garbage collection in Spark interpreter

2016-07-25 Thread Sachin Janani
Hi All, I was looking into the spark interpreter code to understand how it works and found that it is using Spark REPL for executing scala code.As per my understanding spark REPL will not do GC unless we restart the interpreter.So my question here is 1) if we are using Spark interpreter will the va