Thank you both for your detailed replies.
I think we all agree on extending the evaluation framework to handle
recommendation models, and choosing the scalable form of ranking, so
we'll do it that way. For now we will work upon Theodore's PR.
Thanks for giving me the reasons behind the design decision about not
having a separate object for the trained model. I haven't thought about
the implications of pipelines, so I think we should keep the current
design and align our new algorithms to it. Of course, we can bring up a
discussion later and reconsider this design, but I see that it's a
separate issue.
I think it would be good to implement a ScoreMatrixFactorizationRecommender
and a RankingMatrixFactorizationRecommender which both work on a
MatrixFactorizationModel. This model can then either be computed by ALS or
DSGD. This could be controlled by a configuration parameter of the
recommenders.
Do you mean having two different predictors, i.e.
Predictor[ScoreMatrixFactorizationRecommender] and
Predictor[RankingMatrixFactorizationRecommender]?
If I understand right, there should be one common class
MatrixFactorizationModel instead of distinct ALS and DSGD classes, and
it should be a configuration parameter which one to use for training?
I like this idea, as both trainers would require almost the same
configuration. AFAIK there would be an additional 'LearningRate'
parameter for DSGD, but apart from that the two configs are the same.
What do you mean with more "typesafe"? I don't see how returning the
trained model from the fit method gives you more type safety.
I probably used the wrong word here. I simply meant that using a
separate type for the trained model, the type ensures that the trained
model cannot be trained again, while an untrained model cannot be used
for prediction.
Regarding the DSGD algorithm, I think it uses another sampling
mechanism, and we cannot reuse the simple SGD solver. However, we will
make sure not to write duplicate code for the same problem. We've also
noticed, independently from DSGD, that the SGD solver is a GD solver in
reality, but I have not found the related issues and discussion, so
pointing me to them was really useful, thanks!
Cheers,
Gabor
