I remember another way to estimate peak capabilities. This one was used in the late 1970's by Ken Thompson / Belle, but maybe was invented earlier: play handicap matches at different time controls, and fit a quadratic curve to the data. If your program is strong enough, then additional computational effort shows diminishing returns. If your program is too weak, then the performance scales linearly, and you don't see diminishing returns to higher computational effort.
It is also possible to have a peak that falls short of perfection if your program's algorithms are not asymptotically optimal, but that was not a problem for chess programs. Belle was 1100 rating points lower than current programs, and it already showed diminishing returns. I recall that the method gave reasonable results. E.g., I remember estimating that Belle would need at least depth 13 searches to contend for championship caliber. -----Original Message----- From: Brian Sheppard [mailto:sheppar...@aol.com] Sent: Tuesday, March 15, 2016 6:20 PM To: 'computer-go@computer-go.org' <computer-go@computer-go.org> Subject: RE: [Computer-go] AlphaGo & DCNN: Handling long-range dependency >So a small error in the opening or middle game can literally be worth anything >by the time the game ends. These are my estimates: human pros >= 24 points lost, and >= 5 game-losing errors against other human pros. I relayed my experience of a comparable experiment with chess, and how those estimates proved to be loose lower bounds, and it would not surprise me if these estimates are also far from perfection. I urge you to construct a model that you feel embodies important characteristics, and get back to us with your estimates. -----Original Message----- From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Darren Cook Sent: Monday, March 14, 2016 5:15 PM To: computer-go@computer-go.org Subject: Re: [Computer-go] AlphaGo & DCNN: Handling long-range dependency > You can also look at the score differentials. If the game is perfect, > then the game ends up on 7 points every time. If players made one > small error (2 points), then the distribution would be much narrower > than it is. I was with you up to this point, but players (computer and strong humans) play to win, not to maximize the score. So a small error in the opening or middle game can literally be worth anything by the time the game ends. > I am certain that there is a vast gap between humans and perfect play. > Maybe 24 points? Four stones?? 24pts would be about two stones (if each handicap stone is twice komi, e.g. see http://senseis.xmp.net/?topic=2464). The old saying is that a pro would need to take 3 to 4 stones against god (i.e. perfect play). Darren _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
