A minor correction in the GTP ref-score command. Score is not from blacks point of view, but from the point of view of the player who's turn to move it is.
- Don On Mon, 2008-10-13 at 19:14 -0400, Don Dailey wrote: > I made a reference bot and I want someone(s) to help me check it out > with equivalent data from their own program. There are no guarantees > that I have this correct of course. > > Doing 1 million play-outs from the opening position I get the following > numbers for various komi: > > playouts: 1,000,000 > komi: 5.5 > moves: 111,030,705 > score: 0.445677 > > playouts: 1,000,000 > komi: 6.0 > moves: 111,066,273 > score: 0.446729 > > playouts: 1,000,000 > komi: 6.5 > moves: 111,040,546 > score: 0.447138 > > playouts: 1,000,000 > komi: 7.0 > moves: 111,029,204 > score: 0.4333795 > > playouts: 1,000,000 > komi: 7.5 > moves: 111,047,843 > score: 0.421281 > > (I also get a score of 0.524478 for 0.0 komi) > > Score is from blacks point of view. Score is not the score of the > best move of course but the combined average score of all 1 million > play-outs using the stated komi and ranges from zero to one. > > I am going to build a test harness to compare multiple bots side by > side using gtp commands. I made up two private gtp commands to > facilitate this: > > ref-nodes -> return total moves executed in play-outs > (including both pass moves at end of each > play-out.) > > ref-score -> return total win fraction for black. > > NOTE: both commands report stats from last given genmove search. > > > > I hope to get peoples opinion on the following implementation > specification. I'm definitely not a writer, so I need to know if this > very informal spec is enough at least for experienced MC bot authors > or where there are still some ambiguous points. > > > I'm using the following implementation specification: > > ----[ bot implementation specification ]---- > > This is an informal implementation specification document for > writing a simple Monte Carlo Bot program. The idea is to build a bot > like this in ANY language and test it for performance (and > conformity.) Can be used as a general language benchmark but is as much > about the implementation as the language. This specification assumes > some knowledge of go and Monte Carlo go programs. (If you don't like > it, please write a better one for me!) > > > > 1. Must be able to play complete games for comprehensive conformity > testing. > > 2. In the play-out phase, the moves must be chosen in a "uniformly > random" way between legal moves that do not fill 1 point eyes and > obey the simple-ko restriction. > > When a move in the play-out is not possible, a pass is given. > > 3. Play-outs stop after 2 consecutive pass moves, OR when N*N*3 > moves have been completed, except that at least 1 move gets tried > where N is the size of the board. So if the board is 9x9 then > the game is stopped after 9*9*3 = 81*3 = 243 move assuming at > least one move has been tried in the play-outs. > > 4. A 1 point eye is an empty point surrounded by friendly stones > for the side to move. Additionally, we have 2 cases. If the > stone is NOT on any edge (where the corner is an edge) there > must be no more than one diagonal enemy stone. If the point in > question is on the edge, there must be NO diagonal enemy stones. > > 5. Scoring is Chinese scoring. When a play-out completes, the > score is taken accounting for komi and statistics are kept. > > 6. Scoring for game play uses AMAF - all moves as first. In the > play-outs, statistics are taken on moves played during the > play-outs. Statistics are taken only on moves that are played by > the side to move, and only if the move in question is being > played for the first time in the play-out (by either side.) A > win/loss record is kept for these moves. > > 7. The move with the highest statistical win rate is the one > selected for move in the actual game. In the case of moves with > even scores the choice is randomly made between them. > > 8. Pass move are never selected as the final move to play unless no > other non-eye filling move is possible. > > 9. Random number generator is unspecified - your program should > simply pass the "black box" test and possible an optional > additional test which consists of long matches between other > known conforming bots. Your program should score close to 50% > against other "properly implemented" programs. > > 10. Suicide not allowed in the play-outs or in games it plays. > > 11. When selecting moves to play in the actual game (not play-outs) > positional superko is checked and forbidden. > > 12. If stats for a move was never seen in the play-outs, (has a count > of zero) it is ignored for move selection. > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
