A web search turned up a 2 page and an 8 page version. I read the
short one. I agree that it's promising work that requires some follow-
up research.
Now that you've read it so many times, what excites you about it? Can
you envision a way to scale it to larger patterns and boards on modern
hardware?
Sent from my iPhone
On Aug 12, 2009, at 11:14 PM, Michael Williams <michaelwilliam...@gmail.com
> wrote:
After about the 5th reading, I'm concluding that this is an
excellent paper. Is anyone (besides the authors) doing research
based on this? There is a lot to do.
David Silver wrote:
Hi everyone,
Please find attached my ICML paper with Gerry Tesauro on
automatically learning a simulation policy for Monte-Carlo Go. Our
preliminary results show a 200+ Elo improvement over previous
approaches, although our experiments were restricted to simple
Monte-Carlo search with no tree on small boards.
Abstract
In this paper we introduce the first algorithms for efficiently
learning a simulation policy for Monte-Carlo search. Our main idea
is to optimise the balance of a simulation policy, so that an
accurate spread of simulation outcomes is maintained, rather than
optimising the direct strength of the simulation policy. We develop
two algorithms for balancing a simulation policy by gradient
descent. The first algorithm optimises the balance of complete
simulations, using a policy gradient algorithm; whereas the second
algorithm optimises the balance over every two steps of simulation.
We compare our algorithms to reinforcement learning and supervised
learning algorithms for maximising the strength of the simulation
policy. We test each algorithm in the domain of 5x5 and 6x6
Computer Go, using a softmax policy that is parameterised by
weights for a hundred simple patterns. When used in a simple Monte-
Carlo search, the policies learnt by simulation balancing achieved
significantly better performance, with half the mean squared error
of a uniform random policy, and equal overall performance to a
sophisticated Go engine.
-Dave
---
---------------------------------------------------------------------
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
