; Date: Wednesday, 30 March 2016 at 18:59
> To: "computer-go@computer-go.org"
> Subject: Re: [Computer-go] Congratulations to AlphaGo (Statistical
> significance of results)
>
> Hey Simon,
>
> I only now remembered:
>
> we actually experimented on the effect
&
g>"
mailto:computer-go@computer-go.org>>
Date: Wednesday, 30 March 2016 at 18:59
To: "computer-go@computer-go.org<mailto:computer-go@computer-go.org>"
mailto:computer-go@computer-go.org>>
Subject: Re: [Computer-go] Congratulations to AlphaGo (Statistical significa
Or, if it's lopsided far from 1/2, Wilson's is just as good, in my
experience.
On Mar 30, 2016 10:29 AM, "Olivier Teytaud" wrote:
> don't use asymptotic normality with a sample size 5, use Fisher's exact
> test
>
> the p-value for the rejection of
> "P(alpha-Go wins a given game against Lee Sedol
don't use asymptotic normality with a sample size 5, use Fisher's exact test
the p-value for the rejection of
"P(alpha-Go wins a given game against Lee Sedol)<.5"
might be something like 3/16
(under the "independent coin" assumption!)
this is not 0.05, but still quite an impressive result :-)
wi
Hey Simon,
I only now remembered:
we actually experimented on the effect
of making 1 blunder (random move instead of learned/searched move)
in Go and Hex
"Blunder Cost in Go and Hex"
so this might be a starting point for your question
of measuring player strength by measuring
all move strengths
In my original post I put a link to
the relevant section of the MacKay
book that shows exactly how to calculate
the probability of superiority
assuming the game outcome is modelled as
a biased coin toss:
http://www.inference.phy.cam.ac.uk/itila/
I was making the point that for this
and for o
"do they have positive or negative correlation?" intriguing question,
Petri. Intuitively, we might arbitrarily divide the human population
into two groups; one which is discouraged by failure, and the other
which takes the Lady MacBeth attitude of "screw your courage to the
sticking point, and we
Since there are only two possible outcomes it pretty much normal. Actually
binomial which will converge to normal given enough samples
Only thing that cans distort is that consecutive games are not
independent (which
is probably the case but do they have positive or negative correlation?)
2016-03
I think the error here is that the game outcome is not a normaly distributed
random value.
Dmitry
30.03.2016, 12:57, "djhbrown ." :
> Simon wrote: "I was discussing the results with a colleague outside
> of the Game AI area the other day when he raised
> the question (which applies to nearly all
Simon wrote: "I was discussing the results with a colleague outside
of the Game AI area the other day when he raised
the question (which applies to nearly all sporting events,
given the small sample size involved)
of statistical significance - suggesting that on another week
the result might have b
On 22 March 2016 at 21:43, Darren Cook wrote:
< snip >
>
> C'mon DeepMind, put that same version on KGS, set to only play 9p
> players, with the same time controls, and let's get 40 games to give it
> a proper ranking. (If 5 games against Lee Sedol are useful, 40 games
> against a range of playe
On 23.03.2016 15:32, Petr Baudis wrote:
these are beautiful posts.
https://massgoblog.wordpress.com/2016/03/11/lee-sedols-strategy-and-alphagos-weakness/
Before you become too excited, also read my comments on the commentary:
http://www.lifein19x19.com/forum/viewtopic.php?p=200539#p200539
--
Thank you, these are beautiful posts. I enjoyed very much reading
a writeup by Go professional who also took the effort to understand the
principles behind MCTS programs as well as develop a basic intution of
the gameplay artifacts, strengths and weaknesses of MCTS. It also
nicely describes the
FYI. We have translated 3 posts by Li Zhe 6p into English.
https://massgoblog.wordpress.com/2016/03/11/lee-sedols-strategy-and-alphagos-weakness/
https://massgoblog.wordpress.com/2016/03/11/game-2-a-nobody-could-have-done-a-better-job-than-lee-sedol/
https://massgoblog.wordpress.com/2016/03/15/bef
> ...
> Pro players who are not familiar with MCTS bot behavior will not see this.
I stand by this:
>> If you want to argue that "their opinion" was wrong because they don't
>> understand the game at the level AlphaGo was playing at, then you can't
>> use their opinion in a positive way either.
Hi Darren,
"Darren Cook"
> ... But, there were also numerous moves where
> the 9-dan pros said, that in *their* opinion, the moves were weak/wrong.
> E.g. wasting ko threats for no reason. Moves even a 1p would never make.
>
> If you want to argue that "their opinion" was wrong because they don't
"Lucas, Simon M"
> my point is that I *think* we can say more (for example
> by not treating the outcome as a black-box event,
> but by appreciating the skill of the individual moves)
* Human professional players were full of praise for some of
AlphaGo's moves, for instance move 37 in game 2.
> ... we witnessed hundreds of moves vetted by 9dan players, especially
> Michael Redmond's, where each move was vetted.
This is a promising approach. But, there were also numerous moves where
the 9-dan pros said, that in *their* opinion, the moves were weak/wrong.
E.g. wasting ko threats for no
This is somewhat moot - if any moves had been significantly and obviously
weak to any observers, the results wouldn't have been 4-1.
I.e. One bad move out of 5 games would give roughly the same strength
information as one loss out of 5 games; consider that the kibitzing was
being done in real time
I think you are reinforcing Simon's original point; i.e. using a more fine
grained approach to statically approximate AlphaGo's ELO where fine grained
is degree of vetting per move and/or a series of moves. That is a
substantially larger sample size and each sample will have a pretty high
degree of
Behalf Of
Álvaro Begué
Sent: 22 March 2016 17:21
To: computer-go
Subject: Re: [Computer-go] Congratulations to AlphaGo (Statistical significance
of results)
A very simple-minded analysis is that, if the null hypothesis is that AlphaGo
and Lee Sedol are
equally strong, AlphaGo would do as well
Given the minimal sample size, bothering over this question won't amount to
much. I think the proper response is that no one thought we'd see this
level of play at this point in our AI efforts and point to the fact that we
witnessed hundreds of moves vetted by 9dan players, especially Michael
Redmo
another interesting question is to judge the bot's strength
by watching the facial gestures and body language of Lee Sedol
with each move...
On Tue, Mar 22, 2016 at 11:46 AM, Álvaro Begué
wrote:
>
>
> On Tue, Mar 22, 2016 at 1:40 PM, Nick Wedd wrote:
>
>> On 22 March 2016 at 17:20, Álvaro Begué
On Tue, Mar 22, 2016 at 1:40 PM, Nick Wedd wrote:
> On 22 March 2016 at 17:20, Álvaro Begué wrote:
>
>> A very simple-minded analysis is that, if the null hypothesis is that
>> AlphaGo and Lee Sedol are equally strong, AlphaGo would do as well as we
>> observed or better 15.625% of the time. Tha
games would go.
From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of
Álvaro Begué
Sent: 22 March 2016 17:21
To: computer-go
Subject: Re: [Computer-go] Congratulations to AlphaGo (Statistical significance
of results)
A very simple-minded analysis is that, if the null
On 22 March 2016 at 17:20, Álvaro Begué wrote:
> A very simple-minded analysis is that, if the null hypothesis is that
> AlphaGo and Lee Sedol are equally strong, AlphaGo would do as well as we
> observed or better 15.625% of the time. That's a p-value that even social
> scientists don't get exci
A very simple-minded analysis is that, if the null hypothesis is that
AlphaGo and Lee Sedol are equally strong, AlphaGo would do as well as we
observed or better 15.625% of the time. That's a p-value that even social
scientists don't get excited about. :)
Álvaro.
On Tue, Mar 22, 2016 at 12:48 PM
Statistical significance requires a null hypothesis... I think it's
probably easiest to ask the question of if I assume an ELO difference of x,
how likely it's a 4-1 result?
Turns out that 220 to 270 ELO has a 41% chance of that result.
>= 10% is -50 to 670 ELO
>= 1% is -250 to 1190 ELO
My numbers
> I'm not sure if we can say with certainty that AlphaGo is significantly
> better Go player than Lee Sedol at this point. What we can say with
> certainty is that AlphaGo is in the same ballpark and at least roughly
> as strong as Lee Sedol. To me, that's enough to be really huge on its
> own ac
Subject: Re: [Computer-go] Congratulations to AlphaGo (Statistical significance
of results)
> I'm not sure if we can say with certainty that AlphaGo is significantly
> better Go player than Lee Sedol at this point. What we can say with
> certainty is that AlphaGo is in the same ballpar
On Tue, Mar 22, 2016 at 04:00:41PM +, Lucas, Simon M wrote:
> With AlphaGo winning 4 games to 1, from a simplistic
> stats point of view (with the prior assumption of a fair
> coin toss) you'd not be able to claim much statistical
> significance, yet most (me included) believe that
> AlphaGo i
Simon,
There's no argument better than evidence, and no evidence available to us
other than *all* of the games that alphago has played publicly.
Among two humans, a 4-1 result wouldn't indicate any more or less than this
4-1 result, but we'd already have very strong elo-type information about
bot
Hi all,
I was discussing the results with a colleague outside
of the Game AI area the other day when he raised
the question (which applies to nearly all sporting events,
given the small sample size involved)
of statistical significance - suggesting that on another week
the result might have been 4
Here is a tinyURL link to a panel discussion of things AlphaGo that included:
• Oliver Roeder: Senior writer at FiveThirtyEight. All too human.
• David Doshay: Archivist for the American Go Association, co-creator
of SlugGo, a Go-playing computer program.
• Matt Ginsberg:
Helo,
popular culture is growing around AlphaGo's win. Some pieces are
nice: others are, let's say, "special":
Here is the link to a nice Youtube video with an A capella hymnus
(31 seconds) on AlphaGo, performed on 9 GPU ;-)
https://www.youtube.com/watch?v=dh_mfGo183Y
Ingo.
__
Congratulations to AlphaGo and its team! You have done what many of us
could only dream to do and in such short time I may add. This is a truly
historical moment and an amazing achievement for AI research!
I hope this is not the end of Go and only sparks more interest in this
beautiful game. What
On 12.03.2016 22:03, Thomas Wolf wrote:
We currently have no measure at all to judge how safe a winor loss is at
any stage of the game.
We have my theory according to my books for assessing the territorial
and the dynamic aspects (development directions, neutral stones,
statuses (incl. those
On Sun, Mar 13, 2016 at 12:33:55AM +, Gonçalo Mendes Ferreira wrote:
> The single machine version and the 2000 machine version is apparently a
> difference in 150 ELO, so maybe a 32 core instance in Amazon would be
> close enough, costing about 1200$ a month. Maybe it's doable, or for a
> Tokyo
The single machine version and the 2000 machine version is apparently a
difference in 150 ELO, so maybe a 32 core instance in Amazon would be
close enough, costing about 1200$ a month. Maybe it's doable, or for a
Tokyo U. or BGA or AGA to set up something like this with Google permission.
On 13/03
I'd expect this achievement by AlphaGo is very similar to when the first
human ran a 4 minute mile. No one had done it prior. However, right after
Roger Bannister did it, suddenly there were people all over the planet
doing it. Roger Bannister essentially made the possibility real, and then
the psy
At this point I don't doubt that the single machine version is
professional strength which is enough to be used as a tool to analyze
games...
On 2016-03-12 16:24, Lukas van de Wiel wrote:
This would be many thousands of dollars per day. A single game would
be more than a thousand dollars in to
This would be many thousands of dollars per day. A single game would be
more than a thousand dollars in total costs.
I do not think a kickstarter project or so would be successful, as the go
community is simply not *that* big...
On Sun, Mar 13, 2016 at 1:17 PM, Gonçalo Mendes Ferreira
wrote:
> I
It does seem unlikely for DeepMind not to move on to "bigger" things,
but maybe the Go community can make some kind of fundraiser to keep an
instance of AlphaGo playing 24/7? I think there are some websites for
this kind of thing. Someone would be in charge of scheduling time for it
to play pros, o
Oh, I did not say that it would not be beneficial, to AlphaGo, and to the
people playing it, and to the Go community as a whole, but still, it will
have to come from somewhere. Just the electricity bill alone would be
hair-raising.
And the big-scale benefits in prestige and marketing are over, with
On Sat, 12 Mar 2016, Lukas van de Wiel wrote:
And the hardware available for this tournament was tremendous. It remains to be
seen whether the hardware and the people
maintaining it would be available for a longer period. The costs of this are
not to be underestimated. Who would pay it?
Th
And the hardware available for this tournament was tremendous. It remains
to be seen whether the hardware and the people maintaining it would be
available for a longer period. The costs of this are not to be
underestimated. Who would pay it?
Lukas
On Sun, Mar 13, 2016 at 12:20 PM, Clark B. Wierda
On Sat, Mar 12, 2016 at 5:05 PM, Thomas Wolf wrote:
> Having AlphaGo playing exclusively on KGS would be such a boost to KGS!
>
> For sure.
The other Go servers might have their own opinion on that.
Clark
___
Computer-go mailing list
Computer-go@compu
O'Flaherty
Sent: Saturday, March 12, 2016 4:56 PM
To: computer-go@computer-go.org
Subject: Re: [Computer-go] Congratulations to AlphaGo
I think you're correct, Thomas. The challenge is going to be getting ANY
professional to be the one who
"takes handicap stones" for the firs
>
>>> On Sat, 12 Mar 2016, Aja Huang wrote:
>>>
>>> Thanks all. AlphaGo has won the match against Lee
>>> Sedol. But there are still 2 games to play.
>>> Aja
>>>
>>>
omputer-go.org> ] On
Behalf Of Lukas van de Wiel
Sent: Saturday, March 12, 2016 12:14 AM
To: computer-go@computer-go.org
<mailto:computer-go@computer-go.org>
Subject: [Computer-go] Congratulat
ness history being
>> made! Awesome!
>>
>> On Sat, Mar 12, 2016 at 2:17 AM, David Fotland
>> wrote:
>>
>> Tremendous games by AlphaGo. Congrat
Thomas Wolf
Sent: Saturday, March 12, 2016 4:04 PM
To: computer-go@computer-go.org
Subject: Re: [Computer-go] Congratulations to AlphaGo
Chris,
Prompted from a discussion on the computer go email list (and my last email
today) :
We currently have no measure at all to judge how safe a winor loss is
de Wiel
Sent: Saturday, March 12, 2016 12:14 AM
To: computer-go@computer-go.org
Subject: [Computer-go] Congratulations to
AlphaGo
Whoa, what a fight! Well
>
>> Tremendous games by AlphaGo. Congratulations!
>>
>>
>>
>> From: Computer-go [mailto:computer-go-boun...@computer-go.org] On
>> Behalf Of Lukas van de Wiel
>> Sent: Saturday, March 12
Hi Ingo,
I have the manuscript of 2 books with each 100 computer generated problems
which 1st class insei Yutae Seo (Korea) picked out of 20,000 computer
generated problems, working for 5 months on this selection. Many have a tricky
ko status. I am happy to provide them.
Simpler even and more re
Give it the hatsuyo-ron :)
On Mar 12, 2016, at 11:12 AM, Ingo Althöfer wrote:
> Hi Thomas,
>
> Von: "Thomas Wolf"
>> A suggestion for possible future games to be arranged between AlphaGo and
>> strong players:
>>
>> Whoever lost shall be given 1 stone or the equivalent of 1/2 stone handcap
>>
Hi Thomas,
Von: "Thomas Wolf"
> A suggestion for possible future games to be arranged between AlphaGo and
> strong players:
>
> Whoever lost shall be given 1 stone or the equivalent of 1/2 stone handcap in
> the
> next game. Games should continue until each side has won at least once. This
> wa
gt;
>> On Sat, Mar 12, 2016 at 2:17 AM, David Fotland
>> wrote:
>>
>> Tremendous games by AlphaGo. Congratulations!
>>
>>
>>
>> From: Computer-go [mailto:computer-go-boun...@computer-go.org] On
>> Behalf Of Lukas van de Wiel
>>
2016 12:14 AM
To: computer-go@computer-go.org
Subject: [Computer-go] Congratulations to AlphaGo
Whoa, what a fight! Well fought, and well won!
Lukas
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/ma
Congratulations!
>>>
>>>
>>>
>>> *From:* Computer-go [mailto:computer-go-boun...@computer-go.org] *On
>>> Behalf Of *Lukas van de Wiel
>>> *Sent:* Saturday, March 12, 2016 12:14 AM
>>> *To:* computer-go@compute
Dear Ingo,
Thank you for the info. I guess these three games were so silent
that both programs estimated almost correct rate and score (for
the earlier stages, CS could be better than Zen, though :).
Best regards, Hideki
Ingo Althofer:
:
>Hi Hideki,
>
>in a sense this is also a great day
Hi Hideki,
in a sense this is also a great day for your Zen bot.
Some German guy who is fluent in Chinese (Prof. Dr. Marc
Oliver Rieger) followed the game today in a Chinese transmission.
He wrote that the Chinese pro player who was commenting
used commercial Zen to get life evaluations for the p
Dear Aja,
congratulations to you and the whole AlphaGo team for this
awesome performance of your bot!
I fully understand that you take also the last two games seriously.
But, please, do it in such a way that Lee Sedol will be willing
to support you in further matches to come.
Enjoy the day,
Ing
m:* Computer-go [mailto:computer-go-boun...@computer-go.org] *On
>>>> Behalf Of *Lukas van de Wiel
>>>> *Sent:* Saturday, March 12, 2016 12:14 AM
>>>> *To:* computer-go@computer-go.org
>>>> *Subject:* [Computer-go] Congratulations to AlphaGo
half Of *Lukas van de Wiel
>>> *Sent:* Saturday, March 12, 2016 12:14 AM
>>> *To:* computer-go@computer-go.org
>>> *Subject:* [Computer-go] Congratulations to AlphaGo
>>>
>>>
>>>
>>> Whoa, what a fight! Well fought, and well won!
Well done, Aja and all the DeepMind team (including all the "backroom
boys" who've given the reliability on the hardware side).
BTW, I've gained great pleasure seeing you sitting there with the union
jack, representing queen and country; you'll probably receive a
knighthood. :-)
> Thanks all. Alp
Congratulations to DeepMind team! What an excellent full-board
perspective.
Hideki
Lukas van de Wiel:
:
>Whoa, what a fight! Well fought, and well won!
>
>Lukas
> inline file
>___
>Computer-go mailing list
>Computer-go@computer-go.org
>http://co
> *Sent:* Saturday, March 12, 2016 12:14 AM
> *To:* computer-go@computer-go.org
> *Subject:* [Computer-go] Congratulations to AlphaGo
>
>
>
> Whoa, what a fight! Well fought, and well won!
>
> Lukas
>
> ___
> Compute
Tremendous games by AlphaGo. Congratulations!
From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of
Lukas van de Wiel
Sent: Saturday, March 12, 2016 12:14 AM
To: computer-go@computer-go.org
Subject: [Computer-go] Congratulations to AlphaGo
Whoa, what a fight! Well
Whoa, what a fight! Well fought, and well won!
Lukas
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
70 matches
Mail list logo
