The primary purpose of a rating system is to predict the results of future games accurately (this is the usual axiom, at least).
In a one-dimensional rating system, such as Elo, where each player's skill is represented by a single number, it is impossible to have a (non-wacky) system where A is expected to beat B in a two-player match, B is expected to beat C in a two-player match, and C is expected to beat A in a two-player match. So if the players are eccentric in that respect, a one-dimensional rating system is always going to have real problems with accurate predictions. Dan On Fri, Jan 22, 2021 at 10:54 AM uurtamo <uurt...@gmail.com> wrote: > also frankly not a problem for a rating system to handle. > > a rating system shouldn't be tweaked to handle eccentricities of its > players other than the general assumptions of how a game's result is > determined (like, does it allow for "win" and "draw" and "undetermined" or > just "win"). > > s. > > > On Fri, Jan 22, 2021 at 6:29 AM David Wu <lightvec...@gmail.com> wrote: > >> On Fri, Jan 22, 2021 at 8:08 AM Rémi Coulom <remi.cou...@gmail.com> >> wrote: >> >>> You are right that non-determinism and bot blind spots are a source of >>> problems with Elo ratings. I add randomness to the openings, but it is >>> still difficult to avoid repeating some patterns. I have just noticed that >>> the two wins of CrazyStone-81-15po against LZ_286_e6e2_p400 were caused by >>> very similar ladders in the opening: >>> http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733333.sgf >>> http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733301.sgf >>> Such a huge blind spot in such a strong engine is likely to cause rating >>> compression. >>> Rémi >>> >> >> I agree, ladders are definitely the other most noticeable way that Elo >> model assumptions may be broken, since pure-zero bots have a hard time with >> them, and can easily cause difference(A,B) + difference(B,C) to be very >> inconsistent with difference(A,C). If some of A,B,C always handle ladders >> very well and some are blind to them, then you are right that probably no >> amount of opening randomization can smooth it out. >> >> _______________________________________________ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > _______________________________________________ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go >
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
