This was right on the mark! It exposed a lot of misconceptions and wrong thinking about MC and evaluation.
- Don Magnus Persson wrote: > I just want to make some comments about MC evaluation to remove some > common misunderstandings. > > I have seen some complaints about misevaluation such as a program > having 65% chance of winning in a game which is lost and the other way > around. For example arguments has been proposed in line with "since > evaluation is so bad there has to be a better way of doing it". > > I just want to point out one thing: any Winrates except 0% and 100% is > wrong assuming perfect play. 1% and 99% (or anything in between) means > that the program is not sophisticated enough to either a) always play > a winning move in the simulations b) search deep enough to solve the > game proving a win/loss. > > *BUT* > > Having an incorrect evaluation is unimportant, as long as the program > plays the best move. What really matters is which move has the highest > winrate relative to all other candidate moves. > > MC-programs with no knowledge except avoiding eyefilling often > evaluate all positions as being very close to 50%. As soon as one adds > appropriate knowledge there is a much larger range between the best > and worst move on the board. Normally this range is correlated with > the strength of the program. (Be aware that buggy programs might have > even larger ranges though). > > Also with UCT the winrate at the root has little to do with any > objective probability of the program winning. If one look at the > principle variation at the end of the game you will notice a large > difference in winrate at each depth. The winrates at the root change > very slowly even if it is 0% or 100% at the leaves, but still the > relative ordering of the moves at the root is often correct. > > *FINALLY* > > The use of MC-eval and UCT are *completely orthogonal* to using go > knowledge in the program. You can add any kind of knowledge at all > stages to the basic search algorithm and possibly benefit from it. > > The problem is enginering. If your implementation of the knowledge are > too slow the program gets weaker with fixed time limits. The knowledge > you added might make the program weaker for a number of reasons. > > Arguments such as "I saw program X misplay situation Y and therefore > MC-eval is flawed" is just plain wrong. It just mean that specific > program X has a flaw and nothing else. > > What one can argue are "I wrote a program with a nice new method for > evaluation and a new approach to searching that plays situation Y > correctly, and also happens to beat program X all the time using the > same hardware". Until I see that argument, I will continue to beleive > that methods similar to MC-eval and UCT-search are the future of > computer go. > > -Magnus > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
