Re: Flink ML recommender system API

2016-11-10 Thread Gábor Hermann
Hello Theodore, Thanks for your reply. Of course. I would have done that in the first place but I had seen the contribution guideline advising to avoid WIP PRs: "No WIP pull requests. We consider pull requests as requests to merge the referenced code as is into the current stable master bran

Re: Flink ML recommender system API

2016-11-10 Thread Theodore Vasiloudis
Hello Gabor, for this type of issue (design decisions) what we've done in the past with FlinkML is to open a PR marked with the WIP tag and take the discussion there, making it easier for people to check out the code and get a feel of advantages/disadvantages of different approaches. Could you do

Re: Flink ML recommender system API

2016-11-10 Thread Gábor Hermann
Hi all, We have managed to fit the ranking recommendation evaluation into the evaluation framework proposed by Thedore (FLINK-2157). There's one main problem, that remains: we have to different predictor traits (Predictor, RankingPredictor) without a common superclass, and that might be probl

Re: Flink ML recommender system API

2016-10-04 Thread Gábor Hermann
Thank you both for your detailed replies. I think we all agree on extending the evaluation framework to handle recommendation models, and choosing the scalable form of ranking, so we'll do it that way. For now we will work upon Theodore's PR. Thanks for giving me the reasons behind the design

Re: Flink ML recommender system API

2016-10-04 Thread Theodore Vasiloudis
Hello all, Thanks for starting this discussion Gabor you bring up a lot of interesting points. In terms of the evaluation framework I would also favor reworking it in order to support recommendation models. We can either we merge the current PR and use it as a basis, or open a new one. For the f

Re: Flink ML recommender system API

2016-10-04 Thread Till Rohrmann
Hi Gabor, thanks for getting involved in Flink's ML library. Always good to have people working on it :-) Some thoughts concerning the points you've raised inline: On Tue, Oct 4, 2016 at 12:47 PM, Gábor Hermann wrote: > Hey all, > > We've been working on improvements for the recommendation in

Flink ML recommender system API

2016-10-04 Thread Gábor Hermann
Hey all, We've been working on improvements for the recommendation in Flink ML, and some API design questions have come up. Our plans in short: - Extend ALS to work on implicit feedback datasets [1] - DSGD implementation for matrix factorization [2] - Ranking prediction based on a matrix facto