You might be interested in delta-bar-delta algorithm for adapting the gain size (0.99 in your example) http://www.cs.ualberta.ca/~sutton/papers/sutton-92a.pdf Lukasz Lew
On Thu, Jun 26, 2008 at 19:58, Jason House <[EMAIL PROTECTED]> wrote: > I tendto like exponentially weighted moving averages when I need a fading > memory. That keeps storage simple, updates fast, and nearly the same effect > > i.e. > wins = 0.99*wins + latest_result > sims = 0.99*sims + 1 > > Sent from my iPhone > > On Jun 26, 2008, at 2:40 PM, "Ivan Dubois" <[EMAIL PROTECTED]> wrote: > >> This same topic already occured on the list some time ago. >> I think the idea is to "forget" older results. For exemple you can compute >> the win rate based only on the last 500 simulations. Older information may >> not be up to date and will not help much because 500 simulations is enough >> to compute an accurate winrate. >> The problem is that you have to store the result of 500 simulations at >> each node. I think some people reported that it does indeed increase the >> strength of their program. >> >> ----- Original Message ----- From: "Peter Drake" <[EMAIL PROTECTED]> >> To: "Computer Go" <computer-go@computer-go.org> >> Sent: Wednesday, June 25, 2008 5:48 PM >> Subject: [computer-go] UCB/UCT and moving targets >> >> >>> UCB (and hence UCT) would treat the following sequences of wins (1) and >>> losses (0) the same: >>> >>> 01010101010101010101010101010101 >>> 00000000000000001111111111111111 >>> 11111111111111110000000000000000 >>> >>> Clearly, it would be better to favor the second sequence, because that >>> move has done more for us lately. Because the tree is growing, the values >>> of the moves are moving targets. >>> >>> Has anyone done any work dealing with this phenomenon, e.g., somehow >>> giving more weight to more recent playouts? >>> >>> Peter Drake >>> http://www.lclark.edu/~drake/ >>> >>> >>> >>> _______________________________________________ >>> computer-go mailing list >>> computer-go@computer-go.org >>> http://www.computer-go.org/mailman/listinfo/computer-go/ >> >> >> _______________________________________________ >> computer-go mailing list >> computer-go@computer-go.org >> http://www.computer-go.org/mailman/listinfo/computer-go/ > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
