I would enter a bot in a KGS Hahn tournament, of Nick's type (2) if it were held. (If KGS permits round-robin scheduling, I think that would be preferable, but not essential.) I think it would be a lot of fun. Which is not to suggest that the regular KGS tournaments need changing.
The early MCTS bots played to maximize territory but then we evaluated their performance in tournaments that only counted wins and losses. We found that changing the bots to maximize percentage (standing in for probability) of wins led to better performance in those tournaments. I think using MCTS playing to maximize territory works perfectly well, when that's how you intend to measure performance. Things like RAVE might even work better. Here's a question. Suppose I want to measure the strength difference between two different playout policies for my engine. 1) I set up two versions to play Hahn games and evaluate the difference in terms of mean average territory per game. 2) I set them both up to maximize probability of win and evaluate the difference in terms of games won. Which way will give me the answer I need with the fewest games? (Note that games for 2) will be shorter because of resignations.) For weak bots, my experience testing 1) is not encouraging. Blowouts between weak bots are very common, making mean average territory quite a noisy metric. But for strong bots, 1) might be better than 2). I wouldn't be surprised. Using the median might be better than using the mean-I didn't think to try it at the time. But for a KGS Hahn tournament, I would prefer using the mean average territory per game. I like the idea of encouraging the strongest bots to try to swindle away every last stone of their hapless prey. Although my bot would be on the receiving end of this.......hmm. - Dave Hillis -----Original Message----- From: Nick Wedd <n...@maproom.co.uk> To: computer-go <computer-go@computer-go.org> Sent: Tue, Nov 24, 2009 6:42 pm Subject: Re: [computer-go] Re: Hahn system tournament and MC bots. and KGS tournament ? In message <200911242252.09463.alain.baecker...@laposte.net>, Alain Baeckeroot <alain.baecker...@laposte.net> writes >In another thread Nick Wedd wrote: > >> The December KGS bot tournament will be 9x9. I guess that if a >> cluster-Zen competes in that (I am hoping it will), it will be >> unbeatable. >> >> The existing pattern of KGS bot tournaments (see >> http://www.weddslist.com/kgs/future.html) means that the January one >> will also be 9x9, then February and March will both be 19x19. >> ... > > >Is there a possibility for an Hahn tournament on KGS ? >maybe with simplified rules: one point on board is one point in tournament >( (c) R.Jasiek ) The tournaments I run on KGS use the server's tournament manager. This makes my job much easier. But it knows nothing about Hahn scoring. Two things I could do: 1.) Run a tournament manually, telling the operators who their opponents in each round will be, and adding up the score myself. I am not very keen on this, I see to much room for error. 2.) Use the tournament manager, and let it plan the pairings based on its own opinion of who is doing well in the tournament (this won't be too far from reality, Aya will beat WeakBot50K either way). But declare the result based on the total Hahn score of the players. I would prefer (2). I would be willing to hold a tournament like that. Nick -- Nick Wedd n...@maproom.co.uk _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
