George Dahl wrote:
I guess the another question is, what would you need to see a static
evaluator do to be so convinced it was useful that you then built a
bot around it? Would it need to win games all by itself with one ply
lookahead?
Here is one way to look at it: Since a search tends to seek out errors in the evaluation, you must minimize the maximum error that can ever occur. If you do
not, then a very fast computer will search many nodes in the tree, find a place that your evaluator makes a big error, and use that error in it's search
decisions. So how can one test the error that your evaluator has?
As you suggested, you can use the evaluator in a game engine and see how it performs. But that performance is also a function of how big your search is and not
only how good your evaluator is. Another option would be to generate positions and test the accuracy of the evaluator. For this, you would two things:
1) An authority on how strong each position is for black
2) A source of applicable positions to test
For the authority, I would use MoGo. You can make it almost arbitrarily strong
by just waiting longer for it to arrive at an answer (and using more memory,
etc).
As for the source of applicable positions, that's a bit harder, IMO. My first thought was to use random positions since you don't want any bias, but that will
probably result in the evaluation of the position being very near 0.5 much of the time. But I would still try this method since the lack of bias makes it so
attractive. Another option is that you could take random positions from actual games and generate a search tree from that position. From that search tree, you
could select a few random nodes to use as test positions. But now we are back to being coupled to the search parameters -- the same thing we were trying to
avoid by not writing a game engine.
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
