Re: Contribution to Spark MLLib

2014-08-13 Thread Debasish Das
Dennis, If it is PLSA with least square loss then the QuadraticMinimizer that we open sourced should be able to solve it for modest topics (till 1000 I believe)...if we integrate a cg solver for equality (Nocedal's KNITRO paper is the reference) the topic size can be increased much larger than ALS

Re: Contribution to Spark MLLib

2014-06-18 Thread Xiangrui Meng
Denis, I think it is fine to have PLSA in MLlib. But I'm not familiar with the modification you mentioned since the paper is new. We may need to spend more time to learn the trade-offs. Feel free to create a JIRA for PLSA and we can move our discussion there. It would be great if you can share your

Re: Contribution to Spark MLLib

2014-06-18 Thread Jayati
Hello Xiangrui, I am looking at the Spark Issues, but just wanted to know, if it is mandatory for me to work on existing JIRAs before I can contribute to MLLib. Regards, Jayati -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Contribution-to-Spark-MLLib-t

Re: Contribution to Spark MLLib

2014-06-18 Thread Denis Turdakov
Hello everybody, Xiangrui, thanks for the link to roadmap. I saw it is planned to implement LDA in the MLlib 1.1. What do you think about PLSA? I understand that LDA is more popular now, but recent research shows that modifications of PLSA sometimes performs better[1]. Furthermore, the most rece

Re: Contribution to Spark MLLib

2014-06-17 Thread Jayati
Hello Xiangrui, Thanks for sharing the roadmap. I really helped. Regards, Jayati -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Contribution-to-Spark-MLLib-tp7716p7826.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Contribution to Spark MLLib

2014-06-17 Thread Xiangrui Meng
Hi Jayati, Thanks for asking! MLlib algorithms are all implemented in Scala. It makes us easier to maintain if we have the implementations in one place. For the roadmap, please visit http://www.slideshare.net/xrmeng/m-llib-hadoopsummit to see features planned for v1.1. Before contributing new algo