I have considered the very same thing and I think it's a great idea! In fact I implemented the same exact algorithm for a simple go program in C and D for the exact purpose of evaluating D. I even used the same data representation and algorithms for doing things so that it was mostly apples to apples.
I think it's easy to describe and code up a simple algorithm. Uniform random play-outs. For the purpose of the benchmark we could avoid using AMAF or else we would have to make sure it was carefully specified. We could specify the simple eye rule that almost all of us use and make sure all of these things are precisely defined. And then we would need a simple test harness to verify that the program was correct. We could do things like measure the average game length and average win percentage in one or more specified positions. We could create a couple extra GTP commands for reporting these numbers. And we could specify that the programs are ready to play complete games so that they could be tested in real games. But I would suggest that this is not just a language contest, but a programming context too. We should not have just one implementation per language but users would be free to submit their own entry for any language because no two implementations would be the same. The only rule is that it must conform and pass the specified tests and the source code is published with each entry for external verification. Users would be free to take an existing entry, improve it, and then resubmit a new entry based on it. In this way we can probably be more likely to get good implementations represented for each language. We have to make sure we have the details exactly correct. Do we detect ko in the play-outs? Superko or just simple ko? Stuff like that. If we did it language shootout style, we would have to post the source code. And have some rules, such as no assembly modules, etc. - Don On Thu, 2008-10-09 at 22:15 +1300, Stuart A. Yeates wrote: > The topic of which programming language to use has been raised > innumerable times in the >5 years I've been on this list and I've been > backward about coming forward with an opinion because the conversation > seems to generate great deals of heat without much light. > > The http://shootout.alioth.debian.org/ site, however, has lead me to > an interesting idea. The site itself is a set of fully described > algorithms, implementations of the algorithms and a test and reporting > harness. > > If we, as a community, could come up with a sufficiently detailed > description of light playouts algorithm (see the current "Light > simulation : Characteristic values" thread), there is no reason that > this algorithm couldn't join them. > > I suspect that detailing the algorithm sufficiently for non-go players > to implement may be surprising challenging. > > cheers > stuart > > On Thu, Oct 9, 2008 at 12:12 PM, Darren Cook <[EMAIL PROTECTED]> wrote: > >> ATS does the binary-trees test 6.2 times quicker than C++ but 2.7 times > >> slower on k-nucleotide (which seems to be about making hash tables): > > > > Prompted by Isaac, I found the single-core benchmarks (change the "u64q" > > to "u32" in the URLs I posted before to get them, or start at > > http://shootout.alioth.debian.org/u32/ ), ATS does binary trees 1.9 > > times quicker, and k-nucleotide 1.4 times slower. > > > > Darren > > > >> http://shootout.alioth.debian.org/u64q/benchmark.php?test=all〈=ats&lang2=gpp > >> http://shootout.alioth.debian.org/u64q/benchmark.php?test=knucleotide〈=all > >> http://shootout.alioth.debian.org/u64q/benchmark.php?test=knucleotide〈=gpp&id=1 > > > > > > -- > > Darren Cook, Software Researcher/Developer > > http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic > > open source dictionary/semantic network) > > http://dcook.org/work/ (About me and my work) > > http://dcook.org/blogs.html (My blogs and articles) > > _______________________________________________ > > computer-go mailing list > > computer-go@computer-go.org > > http://www.computer-go.org/mailman/listinfo/computer-go/ > > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
