1.4.0 resolves the problem.
The total classes loaded for an updateStateByKey over Int and String types
does not increase.
The total classes loaded for an updateStateByKey over case classes does
increase over time, but
the processing remains stable. Both memory consumption and CPU load remain
boun
Interesting, only in local[*]! In the github you pointed to, what is the
main that you were running.
TD
On Mon, May 25, 2015 at 9:23 AM, rsearle wrote:
> Further experimentation indicates these problems only occur when master is
> local[*].
>
> There are no issues if a standalone cluster is us
Further experimentation indicates these problems only occur when master is
local[*].
There are no issues if a standalone cluster is used.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-updateStateByKey-fails-with-class-leak-when-using-case-classes-r
