On Tue, Mar 22, 2016 at 1:40 PM, Nick Wedd <mapr...@gmail.com> wrote:
> On 22 March 2016 at 17:20, Álvaro Begué <alvaro.be...@gmail.com> wrote: > >> A very simple-minded analysis is that, if the null hypothesis is that >> AlphaGo and Lee Sedol are equally strong, AlphaGo would do as well as we >> observed or better 15.625% of the time. That's a p-value that even social >> scientists don't get excited about. :) >> >> > "For "as well ... or better", I make it 18.75%. > I obviously can't count. :) Thanks for the correction. Álvaro. > > Nick > > > >> Álvaro. >> >> >> On Tue, Mar 22, 2016 at 12:48 PM, Jason House < >> jason.james.ho...@gmail.com> wrote: >> >>> Statistical significance requires a null hypothesis... I think it's >>> probably easiest to ask the question of if I assume an ELO difference of x, >>> how likely it's a 4-1 result? >>> Turns out that 220 to 270 ELO has a 41% chance of that result. >>> >= 10% is -50 to 670 ELO >>> >= 1% is -250 to 1190 ELO >>> My numbers may be slightly off from eyeballing things in a simple excel >>> sheet. The idea and ranges should be clear though >>> On Mar 22, 2016 12:00 PM, "Lucas, Simon M" <s...@essex.ac.uk> wrote: >>> >>>> Hi all, >>>> >>>> I was discussing the results with a colleague outside >>>> of the Game AI area the other day when he raised >>>> the question (which applies to nearly all sporting events, >>>> given the small sample size involved) >>>> of statistical significance - suggesting that on another week >>>> the result might have been 4-1 to Lee Sedol. >>>> >>>> I pointed out that in games of skill there's much more to judge than >>>> just the final >>>> outcome of each game, but wondered if anyone had any better (or worse :) >>>> arguments - or had even engaged in the same type of >>>> conversation. >>>> >>>> With AlphaGo winning 4 games to 1, from a simplistic >>>> stats point of view (with the prior assumption of a fair >>>> coin toss) you'd not be able to claim much statistical >>>> significance, yet most (me included) believe that >>>> AlphaGo is a genuinely better Go player than Lee Sedol. >>>> >>>> From a stats viewpoint you can use this approach: >>>> http://www.inference.phy.cam.ac.uk/itprnn/book.pdf >>>> (see section 3.2 on page 51) >>>> >>>> but given even priors it won't tell you much. >>>> >>>> Anyone know any good references for refuting this >>>> type of argument - the fact is of course that a game of Go >>>> is nothing like a coin toss. Games of skill tend to base their >>>> outcomes on the result of many (in the case of Go many hundreds of) >>>> individual actions. >>>> >>>> Best wishes, >>>> >>>> Simon >>>> >>>> >>>> _______________________________________________ >>>> Computer-go mailing list >>>> Computer-go@computer-go.org >>>> http://computer-go.org/mailman/listinfo/computer-go >>> >>> >>> _______________________________________________ >>> Computer-go mailing list >>> Computer-go@computer-go.org >>> http://computer-go.org/mailman/listinfo/computer-go >>> >> >> >> _______________________________________________ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > > > > -- > Nick Wedd mapr...@gmail.com > > _______________________________________________ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go >
_______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
