Hi Brian!
In my tests with Valkyria I have something like a 4-5% improvement in
winrate against gnugo using ownership. But I think you need to be much
more careful in how you test these things.
Testing on CGOS is a no-no for me, because the opposition changes from
hour to hour, so unless there are large effects in playing strength it
is very hard to detect them on CGOS. I am currently testing against
GnuGo or Fuego or Valkyria itself using twogtp.jar from GoGui. The
nice thing with testing MC programs is that one can set the number of
playouts low and play a lot of games.
You might also consider simplifying your code. Just doing something
simple like this:
CombinedWinrate = AMAFWinRate + k*OwnerShip.
I would then vary k from 0, 0.001, 0.01, 0.1, 1.
Here I am expecting really bad performance for k=1, but I always try
to include some extreme values so that I know for sure that there is
no bug and the results make sense.
I will run 500-2500 games per parameter because often the effects are
really small and needs tons of data to be detected. I learned the hard
way that it is too tempting to make quick conclusion on insufficient
data.
For every test I also need to think hard about how fast the programs
should play. Fast play gives more data, but may not generalize to slow
play on CGOS for example.
When you know how this works, you can start experimenting with more
complex code and more parameters.
That is the philosophy I try to follow for my testing of Valkyria, and
I hope some of it could be helpful.
Quoting Brian Sheppard <sheppar...@aol.com>:
In a paper published a while ago, Remi Coulom showed that 64 MC trials
(i.e., just random, no tree) was a useful predictor of move quality.
In particular, Remi counted how often each point ended up in possession
of the side to move. He then measured the probability of being the best
move as a function of the frequency of possession. Remi found that if
the possession frequency was around 1/3 then the move was most likely
to be best, with decreasing probabilities elsewhere.
I have been trying to extract more information from each trial, since
it seems to me that we are discarding useful information when we use
only the result of a trial. So I tried to implement Remi's idea in a UCT
program.
This is very different from Remi's situation, in which the MC trials are
done before the predictor is used in a tree search. Here, we will have
a tree search going on concurrently with collecting data about point
ownership.
My implementation used the first N trials of each UCT node to collect
point ownership information. After the first M trials, it would use that
information to bias the RAVE statistics. That is, in the selectBest
routine I had an expression like this:
for all moves {
// Get the observed RAVE values:
nRAVE = RAVETrials[move];
wRAVE = RAVEWins[move];
// Dynamically adjust according to point ownership:
if (trialCount < M) {
; // Do nothing.
}
else if (Ownership[move] < 0.125) {
nRAVE += ownershipTrialsParams[0];
wRAVE += ownershipWinsParams[0];
}
else if (Ownership[move] < 0.250) {
nRAVE += ownershipTrialsParams[1];
wRAVE += ownershipWinsParams[1];
}
else if (Ownership[move] < 0.375) {
nRAVE += ownershipTrialsParams[2];
wRAVE += ownershipWinsParams[2];
}
else if (Ownership[move] < 0.500) {
nRAVE += ownershipTrialsParams[3];
wRAVE += ownershipWinsParams[3];
}
else if (Ownership[move] < 0.625) {
nRAVE += ownershipTrialsParams[4];
wRAVE += ownershipWinsParams[4];
}
else if (Ownership[move] < 0.750) {
nRAVE += ownershipTrialsParams[5];
wRAVE += ownershipWinsParams[5];
}
else if (Ownership[move] < 0.875) {
nRAVE += ownershipTrialsParams[6];
wRAVE += ownershipWinsParams[6];
}
else {
nRAVE += ownershipTrialsParams[7];
wRAVE += ownershipWinsParams[7];
}
// Now use nRAVE and wRAVE to order the moves for expansion....
}
The bottom line is that the result was negative. In the test period, Pebbles
won 69% (724 out of 1039) of CGOS games when not using this feature and
less than 59% when using this feature. I tried a few parameter settings.
Far from exhaustive, but mostly in line with Remi's paper.
The best parameter settings showed 59% (110 out of 184, which is 2.4
standard deviations lower). But maybe you can learn from my mistakes
and figure out how to make it work.
I have no idea why this implementation doesn't work. Maybe RAVE does a
good job already of determining where to play, so ownership information
is redundant. Maybe different parameter settings would work. Maybe just
overhead (but I doubt that; the overhead wouldn't account for such a
significant drop).
Anyway, if you try something like this, please let me know how it works out.
Or if you have other ideas about how to extract more information from
trials.
Best,
Brian
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
--
Magnus Persson
Berlin, Germany
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
