On Mon, Dec 15, 2014 at 02:57:32PM -0500, Brian Sheppard wrote:
> I found the 14% win rate against Fuego is potentially impressive, but I
> didn't get a sense for Fuego's effort level in those games. E.g., Elo
> ratings. MCTS actually doesn't play particularly well until a sufficient
> investment is made.
Generally I'd expect Fuego in the described hardware configurations
and time seetings to be in 2k-1d KGS range.
> I am not sure what to think about winning 91% against Gnu Go. Gnu Go makes a
> lot of moves based on rules, so it "replays" games. I found that many of
> Pebbles games against Gnu Go were move-for-move repeats of previous games, so
> much so that I had to randomize Pebbles if I wanted to use Gnu Go for
> calibrating parameters. My guess is that the 91% rate is substantially
> attributable to the way that Gnu Go's rule set interacts with the positions
> that the NN likes. This could be a measure of strength, but not necessarily.
That's an excellent point!
> My impression is that the progressive bias systems in MCTS programs should
> prioritize interesting moves to search. A good progressive bias system might
> have a high move prediction rate, but that will be a side-effect of tuning it
> for its intended purpose. E.g., it is important to search a lot of bad moves
> because you need to know for *certain* that they are bad.
That sounds a bit backwards; it's enough to find a single good move,
you don't need to confirm that all other moves are worse. Of course
sometimes this collapses to the same problem, but not nearly all the
time.
> Similarly, it is my impression is that a good progressive bias engine does
> not have to be a strong stand-alone player. Strong play implies a degree of
> tactical pattern matching that is not necessary when the system's
> responsibility is to prioritize moves. Tactical accuracy should be delegated
> to the search engine. The theoretical prediction is that MCTS search will be
> (asymptotically) a better judge of tactical results.
I don't think anyone would *aim* to make the move predictor as strong
as possible, just that everyone is surprised that it is so strong
"coincidentally". :-)
Still, strong play makes sense for a strong predictor. I believe I
can also beat GNUGo >90% of time in blitz settings without doing pretty
much *any* concious sequence reading. So I would expect a module that's
supposed to mirror my intuition to do the same.
> Finally, I am not a fan of NN in the MCTS architecture. The NN architecture
> imposes a high CPU burden (e.g., compared to decision trees), and this study
> didn't produce such a breakthrough in accuracy that I would give away
> performance.
...so maybe it is MCTS that has to go! We could be in for more
surprises. Don't be emotionally attached to your groups.
--
Petr Baudis
If you do not work on an important problem, it's unlikely
you'll do important work. -- R. Hamming
http://www.cs.virginia.edu/~robins/YouAndYourResearch.html
_______________________________________________
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
