After about the 5th reading, I'm concluding that this is an excellent paper.
Is anyone (besides the authors) doing research based on this? There is a lot
to do.
David Silver wrote:
Hi everyone,
Please find attached my ICML paper with Gerry Tesauro on automatically
learning a simulation policy for Monte-Carlo Go. Our preliminary results
show a 200+ Elo improvement over previous approaches, although our
experiments were restricted to simple Monte-Carlo search with no tree on
small boards.
Abstract
In this paper we introduce the first algorithms for efficiently learning
a simulation policy for Monte-Carlo search. Our main idea is to optimise
the balance of a simulation policy, so that an accurate spread of
simulation outcomes is maintained, rather than optimising the direct
strength of the simulation policy. We develop two algorithms for
balancing a simulation policy by gradient descent. The first algorithm
optimises the balance of complete simulations, using a policy gradient
algorithm; whereas the second algorithm optimises the balance over every
two steps of simulation. We compare our algorithms to reinforcement
learning and supervised learning algorithms for maximising the strength
of the simulation policy. We test each algorithm in the domain of 5x5
and 6x6 Computer Go, using a softmax policy that is parameterised by
weights for a hundred simple patterns. When used in a simple Monte-Carlo
search, the policies learnt by simulation balancing achieved
significantly better performance, with half the mean squared error of a
uniform random policy, and equal overall performance to a sophisticated
Go engine.
-Dave
------------------------------------------------------------------------
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
