On Fri, Jan 22, 2021 at 8:08 AM Rémi Coulom <remi.cou...@gmail.com> wrote:
> You are right that non-determinism and bot blind spots are a source of > problems with Elo ratings. I add randomness to the openings, but it is > still difficult to avoid repeating some patterns. I have just noticed that > the two wins of CrazyStone-81-15po against LZ_286_e6e2_p400 were caused by > very similar ladders in the opening: > http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733333.sgf > http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733301.sgf > Such a huge blind spot in such a strong engine is likely to cause rating > compression. > Rémi > I agree, ladders are definitely the other most noticeable way that Elo model assumptions may be broken, since pure-zero bots have a hard time with them, and can easily cause difference(A,B) + difference(B,C) to be very inconsistent with difference(A,C). If some of A,B,C always handle ladders very well and some are blind to them, then you are right that probably no amount of opening randomization can smooth it out.
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
