Hi! On Wed, Feb 03, 2016 at 10:24:51AM +0100, Robert Jasiek wrote: > AlphaGo is said to predict 57% of professionals' moves. How is this number > measured and from which sample? > > At some turns, there is only one correct move - at other turns, strong go > players would say that there are several valid supposedly correct moves. > This is one of the reasons why 100% cannot be the optimum but a smaller > percentage must be the best. > > Pro players, or players of the database sample (incl. real world 3d players > being 9d on KGS), make mistakes. A neural net learns from a sample and > therefore also learns the mistakes. This is the most important reason why > 100% cannot be the optimum but a smaller percentage must be the best. > > (Roughly) which percentage is optimal? Why? Is the optimum greater or > smaller than 57%?
You are right about these questions. This is from the KGS dataset http://u-go.net/gamerecords/ that contains 7d+ games. Yes, optimum is very far below 100%; there was some discussion on this mailing list in the past about checking strong human performance on this dataset, but AFAIK nothing came out of that yet... My impression is that current neural networks seem to converge to about 60%. It would be interesting if humans can still do better. Another idea would be considering accuracy as a top N measure among selected moves, whether it has better discernive power for currently used models. -- Petr Baudis If you have good ideas, good data and fast computers, you can do almost anything. -- Geoffrey Hinton _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
