Or, if it's lopsided far from 1/2, Wilson's is just as good, in my experience. On Mar 30, 2016 10:29 AM, "Olivier Teytaud" <teyt...@lri.fr> wrote:
> don't use asymptotic normality with a sample size 5, use Fisher's exact > test > > the p-value for the rejection of > "P(alpha-Go wins a given game against Lee Sedol)<.5" > might be something like 3/16 > (under the "independent coin" assumption!) > > this is not 0.05, but still quite an impressive result :-) > > with 5-0 it would have been <0.05. > > > > On Wed, Mar 30, 2016 at 6:59 PM, Ryan Hayward <hayw...@ualberta.ca> wrote: > >> Hey Simon, >> >> I only now remembered: >> >> we actually experimented on the effect >> of making 1 blunder (random move instead of learned/searched move) >> in Go and Hex >> >> "Blunder Cost in Go and Hex" >> >> so this might be a starting point for your question >> of measuring player strength by measuring >> all move strengths... >> >> https://webdocs.cs.ualberta.ca/~hayward/papers/blunder.pdf >> >> On Wed, Mar 30, 2016 at 5:29 AM, Lucas, Simon M <s...@essex.ac.uk> wrote: >> >>> In my original post I put a link to >>> the relevant section of the MacKay >>> book that shows exactly how to calculate >>> the probability of superiority >>> assuming the game outcome is modelled as >>> a biased coin toss: >>> >>> http://www.inference.phy.cam.ac.uk/itila/ >>> >>> >>> I was making the point that for this >>> >>> and for other outcomes of skill-based games >>> we can do so much more (and as humans we intuitively >>> DO do so much more) than just look at the event >>> outcome - and maybe as a community we should do that more >>> routinely and more quantitatively (e.g. >>> by analysing the quality of each move / action) >>> >>> Best wishes, >>> >>> Simon >>> >>> >>> >>> On 30/03/2016, 11:57, "Computer-go on behalf of djhbrown ." < >>> computer-go-boun...@computer-go.org on behalf of djhbr...@gmail.com> >>> wrote: >>> >>> >Simon wrote: "I was discussing the results with a colleague outside >>> >of the Game AI area the other day when he raised >>> >the question (which applies to nearly all sporting events, >>> >given the small sample size involved) >>> >of statistical significance - suggesting that on another week >>> >the result might have been 4-1 to Lee Sedol." >>> > >>> >call me naive, but perhaps you could ask your colleague to calculate >>> >the probability one of side winning 4 games out of 5, and then say >>> >whether that is within 2 standard deviations of the norm. >>> > >>> >his suggestion is complete nonsense, regardless of the small sample >>> >size. perhaps you could ask a statistician next time. >>> > >>> >-- >>> >patient: "whenever i open my mouth, i get a shooting pain in my foot" >>> >doctor: "fire!" >>> >http://sites.google.com/site/djhbrown2/home >>> >https://www.youtube.com/user/djhbrown >>> >_______________________________________________ >>> >Computer-go mailing list >>> >Computer-go@computer-go.org >>> >http://computer-go.org/mailman/listinfo/computer-go >>> _______________________________________________ >>> Computer-go mailing list >>> Computer-go@computer-go.org >>> http://computer-go.org/mailman/listinfo/computer-go >>> >> >> >> >> -- >> Ryan B Hayward >> Professor and Director (Outreach+Diversity) >> Computing Science, UAlberta >> >> _______________________________________________ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > > > > -- > ========================================================= > Olivier Teytaud, olivier.teyt...@inria.fr, TAO, LRI, UMR 8623(CNRS - > Univ. Paris-Sud), > bat 490 Univ. Paris-Sud F-91405 Orsay Cedex France > http://www.slideshare.net/teytaud > > > _______________________________________________ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go >
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
