Amen to that. When using positions to judge the strength of a program, one would need to test not just one "pro move", but a sequence of plays -- including some which don't appear in pro games. A pro knows how to deal decisively not only with the optimal plays of other pros, but also with suboptimal plays from the rest of us. Programs are often even stranger than human players.
If I were designing a test set, I'd ask pros to defeat the program, and would convert the blunders into a test set. To improve, the program would have to generalize the lessons learned from those test cases. Terry McIntyre <terrymcint...@yahoo.com> -- People never lie so much as after a hunt, during a war or before an election. - Otto von Bismarck ________________________________ From: steve uurtamo <uurt...@gmail.com> To: computer-go <computer-go@computer-go.org> Sent: Tuesday, April 7, 2009 5:12:27 PM Subject: Re: [computer-go] Fast ways to evaluate program strength. otherwise pair-go wouldn't be as funny to watch. s. On Tue, Apr 7, 2009 at 8:05 PM, Michael Williams <michaelwilliam...@gmail.com> wrote: > Łukasz Lew wrote: >> >> I would like to rephrase my question: >> Let's measure prediction of pro moves of a whole engine while >> modifying heavy playouts / MCTS in the engine. >> How well might it work? > > Probably not well. Because what matters is not how often you play strong > moves, but how often you avoid blunders. > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
