On Sat, 2008-11-29 at 11:58 +0100, Denis fidaali wrote: > > From my own experience, an important testing case whenever trying > to get AMAF to work, is the scaling study. >
No truer words ever spoken. This is one of the secrets to strong programs, if they scale, they are probably soundly designed. I do that with Chess. I find that some program changes scale up, particular sound algorithms that reduce the branching factor. I have to run tests pretty fast in order to get results I can interpret, but I also plot the result visually with gnuplot. As many here will recall, my own Fatman study vs Leela showed that Leela scaled better with increasing depth than Fatman. Nothing like a graph to reveal this very clearly, although you can also look at the numbers if you are careful. It's important to point out that you will be completely mislead if you don't get enough samples. It's very rare that 100 or 200 games are enough to draw any conclusions (unless it's really lopsided.) I remember once thinking I had found a clear scalable improvement but decided that it must run longer - but I was hopeful. When the improvement started to decline, I discovered that I had by accident been running the same exact program against itself. The point is that it is not uncommon to get really "lucky", and have an equal program look substantially superior - for a while. - Don > My prototype was quite strong considering that it used only 1000 > light playout > (and score 25-30 % win against gnugo lvl 0), but it seemed to not get > much > over that as the number of playout grew ... (also there had a serious > exponential complexity problem, which i never get into the trouble of > investigating :) ) > > I know that Zoe was about 2000 elo with i think 50k simulations, > and ... never > got any real better as the number of simulations increased. > > Both prototype were toying with AMAF, so i really think you need a bit > of scalability > study whenever trying to asses an engine employing it. (although it > could very well be > that the scalability trouble came out of some nasty bugs. Both > aforementioned prototype > where quite messy ...) > > > From: [EMAIL PROTECTED] > > Subject: Re: [computer-go] RAVE formula of David Silver (reposted) > > Date: Sat, 29 Nov 2008 03:39:58 -0200 > > To: computer-go@computer-go.org > > CC: > > > > > > On 28-nov-08, at 17:28, [EMAIL PROTECTED] wrote: > > > > > I would be very interested to see the RAVE code from Valkyria. > I'm > > > sure others would be too. > > > > > > > I'm much more interested in a general concise description. If such > a > > description cannot be given easily, then I think there's little > point > > including it in the definition of a MCTS reference engine. > > > > I found a serious flaw in my code collecting the AMAF scores, which > > explains why I wasn't seeing any gains so far with AMAF turned on. > > Now over the first 100+ games UCT+RAVE scores 90% over plain UCT. > I'm > > going to run a test overnight, but so far it looks good. It should > > have collected a few thousand samples by tomorrow. > > > > Hopefully next week I can go back to testing my list of playout > > improvements, which is why I started making the MCTS reference > > implementation in the first place. This RAVE stuff caused a bit of > a > > distraction, but it's nice to have if it works. > > > > Mark > > _______________________________________________ > > computer-go mailing list > > computer-go@computer-go.org > > http://www.computer-go.org/mailman/listinfo/computer-go/ > > > ______________________________________________________________________ > Souhaitez vous « être au bureau sans y être » ? Oui je le veux ! > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
