On Thu, Apr 9, 2015 at 5:50 AM, Niklas Wenzel <nikwen.develo...@gmail.com> wrote:
Thanks for bringing this up! I completely agree that the current sorting is naive at best and it is not intended to be the way we sort search results at all, just a convenient way to get us bootstrapped. > To me it seems that we agree we should improve the sorting algorithm. I > acknowledge the problem of having no way to deal with unfair reviews but I > believe this should overall be separated from the sorting problem, even > though a better sorting mechanism might relativise some bad reviews. > > Until now, we've had two suggestions: the Bayesian average and the solution > mentioned by Selene, where each app would get a 3 star rating by default > which would be adjusted later when people start reviewing the application. > In my opinion these are pretty good solutions, with each of them having > their individual advantages or disadvantages. > The Bayesian average tries to predict what would happen if plenty of people > were asked to rate the app but apps without bad reviews only still rank > better than those without any reviews. > Selene's method solves the latter, but depending on the implementation it > might be disadvantageous for apps with few reviews. It would be nice to see > a code example here. Selene, is your music player open source? There are a lot of good points about people gaming the system and problems with different approaches. Luckily, this is not a new problem at all. In fact, it's not even new to Ubuntu! Matthew Paul Thomas spent some time thinking about this years ago and put together a great wiki page[1]. Which points to a great post on how to avoid the most common problems: http://www.evanmiller.org/how-not-to-sort-by-average-rating.html We also plan to add in other metrics over time that are harder to game: # of downloads normalised by download date, retention (how many people keep it installed?), usage (opt-in feature to let the store know how often the app is used), quality (number of crashes) and others. To try and avoid loops where apps get more downloads because they started off higher on the list, climbing up higher and making it harder for new or less well-known apps to surface, we'll likely introduce a small amount of serendipity into the search results, occasionally returning something high on the list that might not normally make it. So yes, we will improve this soon and improve it iteratively over time, I expect us to be working on this on and off as long as the store exists. > My next question: I'd love to help implement the new ranking mechanism. Is > the code for the app store backend available somewhere so that I can > retrieve and modify it? If not, who is the person to talk to? Unfortunately this piece isn't open source. The obscurity of how search results are (will be) sorted in this case will help make it harder to game the system by developers, making it a bit of a better experience for users. A bit down the line, though, we might be able to open up some amount of data and have a contest to improve sorting, similar to what Netflix has done in the past[2]. In the mean time we'll soon improve the sorting a bit to get us through this next phase of growth, which is a fantastic problem to have :) [1] https://wiki.ubuntu.com/SoftwareCenter/RatingsAndReviews [2] http://en.wikipedia.org/wiki/Netflix_Prize -- Martin -- Mailing list: https://launchpad.net/~ubuntu-phone Post to : ubuntu-phone@lists.launchpad.net Unsubscribe : https://launchpad.net/~ubuntu-phone More help : https://help.launchpad.net/ListHelp
