StackOverflow Error when run ALS with 100 iterations

[Xiaoli Li]

Hi,

I am testing ALS using 7 nodes. Each node has 4 cores and 8G memeory. ALS
program cannot run  even with a very small size of training data (about 91
lines) due to StackVverFlow error when I set the number of iterations to
100. I think the problem may be caused by updateFeatures method which
updates products RDD iteratively by join previous products RDD.


I am writing a program which has a similar update process with ALS.  This
problem also appeared when I iterate too many times (more than 80).

The iterative part of my code is as following:

solution = outlinks.join(solution). map {
     .......
 }


Has anyone had similar problem?  Thanks.


Xiaoli