I have two versions of the reference bot. A C and a Java version. I am doing a conformance self-test on my java and C versions and they do not agree. The simple stats seem to agree but they play at different strengths when self-testing with actual games. The C version is playing about 40 ELO weaker and it's less and less likely to be statistical noise.
So I found a minor bug, fixed it, and run the tests again and still find an error, but it looks to be around 25 ELO after a few thousand games. There are enough games that this bias looks consistent. The weaker C version, uses the Mersenne Twister which I have some confidence in, so I really doubt the RNG is a factor here since the C program probably has the superior RNG. I am also careful that I select numbers in a uniformly random way - a lot of people do this wrong because stuff like (rng() % N) doesn't do what people think it does. So I have a little mystery here. The program is pretty straightforward and simple, so it proves something that the chess people know, that the strength of your program is heavily dependent on how many bugs you have and that straightforward bug-free programs tend to be pretty strong. It has been said that the secret of Fruit, a very strong chess program is mostly that it is bug-free, there is nothing spectacular about how it was written other than this. I don't know to what extent that is true for fruit, it's just something that has been spread around, but I know that it tends to be true for me and my software. Even if I find the bug(s) and fix it, I am still comparing two of my own implementations. So it would be good if someone else would send me their "conforming" version (for linux preferably, although I can usually run exe files) and of course my java implementation is freely available. I could also send a compiled java version (that does check out against the JIT version), a binary for 64 bit linux. That version is actually faster than the latest IBM java JIT by a good little bit and it has a smaller memory footprint, faster startup time, etc. It almost makes Java a practical language. - Don
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
