Ingo, I'm not a proper statistician, but I believe there's a crucial second step that's missing in your analysis of significance. Even if this were the only computer-go test that you personally had ever conducted, we would nevertheless need to take into account all of the other tests being conducted within the community. On any given day, some high number of similar tests are carried out by members of this list. They are testing different hypotheses to be sure, but that doesn't get us off the hook at all.
What it boils down to is this: how frequently does *somebody* get a 95% confidence result about *something* that isn't going to hold up under further testing? This issue comes up all the time in epidemiology (e.g. cancer clusters near power lines), medical studies, bioinformatics, etc.. - Dave Hillis -----Original Message----- From: "Ingo Althöfer" <3-hirn-ver...@gmx.de> To: computer-go@computer-go.org Sent: Thu, Feb 18, 2010 7:28 am Subject: [computer-go] Re: Dynamic Komi at 9x9 ? Hello Don, everal very good points by you! Does anyone have data based on several thousands games that attempts to measure the effect of dynamic komi? I would like to see results that are statistically meaningful. I had eight handplayed (4 + 4) games on 19x19 with very igh handicap, where the version with dynamic komi (rule 42) ained a 3-1 score and the version with static komi erformed 0-4 versus the same opponent. This is evidence n the 95% region that the version with dynamic komi is ot weaker than the static version. > We need to see a few thousand games played A few hundreds or even a few dozens may be sufficient when he outcome is very clear. > against a fixed opponent WITH dynamic komi, and then the same program without dyanmic komi playing against the same opponent with the same number of games. The number of games must be decided before the test is run, or the error margin calculation is meaningless. I am willing to provide the statistical part, when programmers un the experiments. As far as I can tell, nobody has yet to produce anything more than anecdotal evidence that this works. I have. See the 4 + 4 games mentioned above, layed with my "rule 42". > Having a person manually adjusting this after every game is completely non-sceientific, unless they are doing it in a fixed way with no decision making on their part Right. > and they are playing thousands of games (or at least enough to get statistically significant results.) Right, especially also the bracket part of your sentence. > I'm not trying to rain on anyone's parade, but I cannot understand why no one has produced a statistically meaningful result on this subject - I would have. Unfortunately I am not a programmer, and am also ot fit in modifying a program code to include dynamic komi. But, to repeat it, I am willing to do statistical home ork. > I am genuinely interested in this since I never was able to make it work when I spent about one intense week on it. (I did not do this with handicap games, but with normal games.) Your sentence in brackets is crucial. I only proposed to use ynamic komi in games with high handicap. Especially I had in ind the situation where the stronger side (giving high handicap) s MC-based. Perhaps, 9x9 instead of 19x19 makes it easier for some programmer o start test series with dynamic komi. Ingo. -- icherer, schneller und einfacher. Die aktuellen Internet-Browser - etzt kostenlos herunterladen! http://portal.gmx.net/de/go/atbrowser ______________________________________________ omputer-go mailing list omputer...@computer-go.org ttp://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
