Re: SPARK OWLQN Exception: Iteration Stage is so slow

2014-07-29 Thread Xiangrui Meng
Do you mind sharing more details, for example, specs of nodes and data size? -Xiangrui 2014-07-29 2:51 GMT-07:00 John Wu : > Hi all, > > > > There is a problem we can’t resolve. We implement the OWLQN algorithm in > parallel with SPARK, > > We don’t know why It is very slow in every iteration stag

SPARK OWLQN Exception: Iteration Stage is so slow

2014-07-29 Thread John Wu
Hi all, There is a problem we can’t resolve. We implement the OWLQN algorithm in parallel with SPARK, We don’t know why It is very slow in every iteration stage, but the load of CPU and Memory of each executor are so low that it seems impossible to make the the every step slow. And there