[computer-go] an idea for a new measure of a computer go program's rank.
Hi, The challenge to write a go playing program that could beat a professional was issued before the wide availability of Internet Go Servers, and broadband access. Under these new conditions, it is trivial to write such a program, provided the game takes place on a server, and at time limits chosen by the program. For example a random point playing program could choose time limits of half a second per move, sudden death. Therefore I suggest that a program's strength can (if needed) be expressed as the shortest time limits that a player of a standard strength (eg Pro. 1 dan) would be willing to play the program at, given an equal reward/loss regime (ie the chance of either winning would be 0.5). The format of time limits for such games would need to be standardised, for example - it could be decided that only limits of the type 'sudden death, x number of seconds per move' were allowed. In that case, 'x' could be used as a measure of the program's strength (as an abreviation for 'would beat a standard strength player half the time at x seconds per move') Of course the strength of a 'standard strength' Go player varies, and professional one dans would likely be unwilling to be beaten in ultra blitz games for the benefit of computer go programmers, so 'amateur 1 dan' is a realistic idea for a standard strength go player. dan ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] KGS hoover perl script and related bash scripts
Hi, I have written a perl program that plays Go (poorly). It uses pattern matching. Patterns are automatically read into a database from sgf files. I will release the source code soon, but first here are some tools that can be used to download and extract games from KGS. They are acceptable to William Shubert, provided there is a 3-4 second delay in requesting a player's index page. This delay has been implemented, so please don't remove it from kgshoover.pl Perl version is 5.8.8, OS is linux. You may need to make some changes for the scripts to work on Windows. Files attached. Use anyway you please. D.Gilder kgs.tar.gz Description: application/tgz ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [Computer-go] Nice graph
How did they do it ? Is there a video of the presentation somewhere ? Thanks On Fri, Mar 25, 2016 at 5:59 PM, David Ongaro wrote: > That would mean 3 stones if the "4 stone handicap" has the same definition > as in the paper (7.5 Komi for white and 3 extra moves for black after the > first move. Yes that implies that a traditional 4 stone handicap (without > Komi for white) is in fact 3.5 Stones). > > > > On 25 Mar 2016, at 17:23, Rémi Coulom wrote: > > > > AlphaGo improved 3-4 stones: > > > > http://i.imgur.com/ylQTErVl.jpg > > > > (Found in the Life in 19x19 forum) > > > > Rémi > > ___ > > Computer-go mailing list > > Computer-go@computer-go.org > > http://computer-go.org/mailman/listinfo/computer-go > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] mcts and tactics
Hello all, It is known that MCTS's week point is tactics. How is AlphaZero able to resolve Go tactics such as ladders efficiently? If I recall correctly many people were asking the same question during the Lee Sedo match -- and it seemed it didn't have any problem with ladders and such. In chess and shogi, there is lots of tactics and plain MCTS as used in AlphaZero shouldn't perform well (one would expect), but apparently AlphaZero didn't seem to have a problem in that regard against stockfish. First of all, I think that AlphaZero is resolving tactics by growing its MCTS tree very rapidly (expand after each visit) -- some people thought initially that NN may have some tactics in it but I don't believe it can do better than a quiescence_search. Tactics requires precise calculations with moves that maynot make sense (sacrfice) -- apparently AlphaZero's positional understanding led it to be superior in this regard as well. My simple MCTS chess engine (single thread) is now better in tactics than it used to be (after removing the rollouts), but it is still far far from the tactical ability of alpha-beta engines with LMR+nullmove. What do you think is AlphaZero's tactical strength coming from ? I am guessing parallel MCTS with larger exploration coefficient for each thread -- this should explore enough not so good moves closer to the root not to miss sshallow tactics. I just wanted to know the opinions of the MCTS experts here.. regards, Daniel ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Mcts and tactics
Thanks! It would be interesting to see the performance of the policy network alone in chess and shogi too. There is no such plot in the arxiv paper. Honestly, I don't expect it to be that good since engines without a lookahead search never performed that well in these domains -- unlike Go which has GnuGo, a very strong player before the MC era. Daniel On Tue, Dec 19, 2017 at 11:57 PM, patrick.bardou via Computer-go < computer-go@computer-go.org> wrote: > Hi Daniel, > > AGZ paper: greedy player based on policy network (= zero look-ahead) has > an estimated ELO of 3000 ~ Fan Hui 2p. > > Professional player level with Zero look-ahead. For me, it is the other > striking aspect of 'Zero' ! ;-) > > IMO, this implies that the NN has indeed captured lots of tactics. Even if > tactics may not be as important in go as in chess, it still matters a lot, > not just in capturing races. It is often at the foundation of the value of > a position (e.g.: life & death status of a group; "value of this position > is X because there exist sequences such that this black group can either > live or link"). > > Hard to imagine 2p level without a great deal of tactics, just strong > positional judgment. Practicaly, for MCTS guided by policy and value > networks, this means the policy networks has to assign good prior to > tactical moves. > > BR, > Patrick > > > > Message d'origine > De : computer-go-requ...@computer-go.org > Date : 20/12/2017 01:57 (GMT+01:00) > À : computer-go@computer-go.org > Objet : Computer-go Digest, Vol 95, Issue 24 > > > Message: 1 > Date: Tue, 19 Dec 2017 16:26:00 -0700 > From: Dan > To: computer-go@computer-go.org > Subject: [Computer-go] mcts and tactics > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > Hello all, > > It is known that MCTS's week point is tactics. How is AlphaZero able to > resolve Go tactics such as ladders efficiently? If I recall correctly many > people were asking the same question during the Lee Sedo match -- and it > seemed it didn't have any problem with ladders and such. > > In chess and shogi, there is lots of tactics and plain MCTS as used in > AlphaZero shouldn't perform well (one would expect), but apparently > AlphaZero didn't seem to have a problem in that regard against stockfish. > First of all, I think that AlphaZero is resolving tactics by growing its > MCTS tree very rapidly (expand after each visit) -- some people thought > initially that NN may have some tactics in it but I don't believe it can do > better than a quiescence_search. Tactics requires precise calculations with > moves that maynot make sense (sacrfice) -- apparently AlphaZero's > positional understanding led it to be superior in this regard as well. > > My simple MCTS chess engine (single thread) is now better in tactics than > it used to be (after removing the rollouts), but it is still far far from > the tactical ability of alpha-beta engines with LMR+nullmove. What do you > think is AlphaZero's tactical strength coming from ? I am guessing parallel > MCTS with larger exploration coefficient for each thread -- this should > explore enough not so good moves closer to the root not to miss sshallow > tactics. > > I just wanted to know the opinions of the MCTS experts. > > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] 9x9 is last frontier?
Hello Aja, Could you enlighten me on how AlphaZero handles tactics in chess ? It seems the mcts approach as described in the paper does not perform well enough. Leela-chess is not performing well enough even though leela-go seems to be doing well. Daniel On Fri, Mar 2, 2018 at 4:52 AM, Aja Huang wrote: > > > 2018-03-02 6:50 GMT+00:00 "Ingo Althöfer" <3-hirn-ver...@gmx.de>: > >> Von: "David Doshay" >> > Go is hard. >> > Programming is hard. >> > >> > Programming Go is hard squared. >> > ;^) >> >> And that on square boards. >> Mama mia! >> > > Go is hard for humans, but in my own opinion I think Go seems to be too > easy for deep learning. So is programming Go now. :) > > Aja > > >> >> ;-) Ingo. >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] 9x9 is last frontier?
Leela chess is here https://github.com/glinscott/leela-chess It uses the exact MCTS algorithm as described in AlphaZero, with value and policy networks, but performs really badly in tactics (often missing 2-3 ply shallow tactics) To get a somewhat strong MCTS chess engine, I had to use alpha-beta rollouts instead, as described in https://www.microsoft.com/en-us/research/wp-content/uploads/2014/11/huang_rollout.pdf I think this could be suitable for 9x9 Go too. It has been a mystery for me how AlphaZero avoided into falling for shallow traps in chess. Daniel On Fri, Mar 2, 2018 at 10:22 AM, Xavier Combelle wrote: > Where is leela chess. How many games it is trained on? > > Le 2 mars 2018 18:20, "Dan" a écrit : > >> Hello Aja, >> >> Could you enlighten me on how AlphaZero handles tactics in chess ? >> >> It seems the mcts approach as described in the paper does not perform >> well enough. >> >> Leela-chess is not performing well enough even though leela-go seems to >> be doing well. >> >> Daniel >> >> >> >> >> >> On Fri, Mar 2, 2018 at 4:52 AM, Aja Huang wrote: >> >>> >>> >>> 2018-03-02 6:50 GMT+00:00 "Ingo Althöfer" <3-hirn-ver...@gmx.de>: >>> >>>> Von: "David Doshay" >>>> > Go is hard. >>>> > Programming is hard. >>>> > >>>> > Programming Go is hard squared. >>>> > ;^) >>>> >>>> And that on square boards. >>>> Mama mia! >>>> >>> >>> Go is hard for humans, but in my own opinion I think Go seems to be too >>> easy for deep learning. So is programming Go now. :) >>> >>> Aja >>> >>> >>>> >>>> ;-) Ingo. >>>> ___ >>>> Computer-go mailing list >>>> Computer-go@computer-go.org >>>> http://computer-go.org/mailman/listinfo/computer-go >>>> >>> >>> >>> ___ >>> Computer-go mailing list >>> Computer-go@computer-go.org >>> http://computer-go.org/mailman/listinfo/computer-go >>> >> >> >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] 9x9 is last frontier?
Actually prior to this it was trained with hundreds of thousands of stockfish games and didn’t do well on tactics (the games were actually a blunder fest). I believe this is a problem of the MCTS used and not due to for lack of training. Go is a strategic game so that is different from chess that is full of traps. I m not surprised Lela zero did well in go. On Mon, Mar 5, 2018 at 2:16 AM Gian-Carlo Pascutto wrote: > On 02-03-18 17:07, Dan wrote: > > Leela-chess is not performing well enough > > I don't understand how one can say that given that they started with the > random network last week only and a few clients. Of course it's bad! > That doesn't say anything about the approach. > > Leela Zero has gotten strong but it has been learning for *months* with > ~400 people. It also took a while to get to 30 kyu. > > -- > GCP > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] 9x9 is last frontier?
I am pretty sure it is an MCTS problem and I suspect not something that could be easily solved with a policy network (could be wrong hree). My opinon is that DCNN is not a miracle worker (as somebody already mentioned here) and it is going to fail resolving tactics. I would be more than happy with it if it has same power as a qsearch to be honest. Search traps are the major problem with games like Chess, and what makes transitioning the success of DCNN from Go to Chess non trivial. The following paper discusses shallow traps that are prevalent in chess. ( https://www.aaai.org/ocs/index.php/ICAPS/ICAPS10/paper/download/1458/1571 ) They mention traps make MCTS very inefficient. Even if the MCTS is given 50x more time is needed by an exhaustive minimax tree, it could fail to find a level-5 or level-7 trap. It will spend, f.i, 95% of its time searching an asymetric tree of depth > 7 when a shallow trap of depth-7 exists, thus, missing to find the level-7 trap. This is very hard to solve even if you have unlimited power. The plain MCTS as used by AlphaZero is the most ill-suited MCTS version in my opinion and i have hard a hard time seeing how it can be competitive with Stockfish tactically. My MCTS chess engine with AlphaZero like MCTS was averaging was missing a lot of tactics. I don't use policy or eval networks but qsearch() for eval, and the policy is basically choosing which ever moves leads to a higher eval. a) My first improvement to the MCTS is to use minimax backups instead of averaging. This was an improvmenet but not something that would solve the traps b) My second improvment is to use alphabeta rollouts. This is a rollouts version that can do nullmove and LMR etc... This is a huge improvment and none of the MCTS versons can match it. More on alpha-beta rollouts here ( https://www.microsoft.com/en-us/research/wp-content/uploads/2014/11/huang_rollout.pdf ) So AlphaZero used none of the above improvements and yet it seems to be tactically strong. Leela-Zero suffered from tactical falls left and right too as I expected. So the only explanation left is the policy network able to avoid traps which I find hard to believe it can identify more than a qsearch level tactics. All I am saying is that my experience (as well as many others) with MCTS for tactical dominated games is bad, and there must be some breakthrough in that regard in AlphaZero for it to be able to compete with Stockfish on a tactical level. I am curious how Remi's attempt at Shogi using AlphaZero's method will turnout. regards, Daniel On Tue, Mar 6, 2018 at 9:41 AM, Brian Sheppard via Computer-go < computer-go@computer-go.org> wrote: > Training on Stockfish games is guaranteed to produce a blunder-fest, > because there are no blunders in the training set and therefore the policy > network never learns how to refute blunders. > > > > This is not a flaw in MCTS, but rather in the policy network. MCTS will > eventually search every move infinitely often, producing asymptotically > optimal play. But if the policy network does not provide the guidance > necessary to rapidly refute the blunders that occur in the search, then > convergence of MCTS to optimal play will be very slow. > > > > It is necessary for the network to train on self-play games using MCTS. > For instance, the AGZ approach samples next states during training games by > sampling from the distribution of visits in the search. Specifically: not > by choosing the most-visited play! > > > > You see how this policy trains both search and evaluation to be internally > consistent? The policy head is trained to refute the bad moves that will > come up in search, and the value head is trained to the value observed by > the full tree. > > > > *From:* Computer-go [mailto:computer-go-boun...@computer-go.org] *On > Behalf Of *Dan > *Sent:* Monday, March 5, 2018 4:55 AM > *To:* computer-go@computer-go.org > *Subject:* Re: [Computer-go] 9x9 is last frontier? > > > > Actually prior to this it was trained with hundreds of thousands of > stockfish games and didn’t do well on tactics (the games were actually a > blunder fest). I believe this is a problem of the MCTS used and not due to > for lack of training. > > > > Go is a strategic game so that is different from chess that is full of > traps. > > I m not surprised Lela zero did well in go. > > > > On Mon, Mar 5, 2018 at 2:16 AM Gian-Carlo Pascutto wrote: > > On 02-03-18 17:07, Dan wrote: > > Leela-chess is not performing well enough > > I don't understand how one can say that given that they started with the > random network last week only and a few clients. Of course it's bad! > That doesn't say anything about the approach. > > Leela Zero has gotten strong but it has been learning for *months
Re: [Computer-go] 9x9 is last frontier?
I did a quick test with my MCTS chess engine wth two different
implementations.
A standard MCTS with averaging, and MCTS with alpha-beta rollouts. The
result is like a 600 elo difference
Finished game 44 (scorpio-pmcts vs scorpio-mcts): 1/2-1/2 {Draw by 3-fold
repetition}
Score of scorpio-mcts vs scorpio-pmcts: 41 - 1 - 2 [0.955] 44
Elo difference: 528.89 +/- nan
scorpio-mcts uses alpha-beta rollouts
scorpio-pmcts is "pure" mcts with averaging and UCB formula.
Daniel
On Tue, Mar 6, 2018 at 11:46 AM, Dan wrote:
> I am pretty sure it is an MCTS problem and I suspect not something that
> could be easily solved with a policy network (could be wrong hree). My
> opinon is that DCNN is not
> a miracle worker (as somebody already mentioned here) and it is going to
> fail resolving tactics. I would be more than happy with it if it has same
> power as a qsearch to be honest.
>
> Search traps are the major problem with games like Chess, and what makes
> transitioning the success of DCNN from Go to Chess non trivial.
> The following paper discusses shallow traps that are prevalent in chess. (
> https://www.aaai.org/ocs/index.php/ICAPS/ICAPS10/paper/download/1458/1571
> )
> They mention traps make MCTS very inefficient. Even if the MCTS is given
> 50x more time is needed by an exhaustive minimax tree, it could fail to
> find a level-5 or level-7 trap.
> It will spend, f.i, 95% of its time searching an asymetric tree of depth >
> 7 when a shallow trap of depth-7 exists, thus, missing to find the level-7
> trap.
> This is very hard to solve even if you have unlimited power.
>
> The plain MCTS as used by AlphaZero is the most ill-suited MCTS version in
> my opinion and i have hard a hard time seeing how it can be competitive
> with Stockfish tactically.
>
> My MCTS chess engine with AlphaZero like MCTS was averaging was missing a
> lot of tactics. I don't use policy or eval networks but qsearch() for eval,
> and the policy is basically
> choosing which ever moves leads to a higher eval.
>
> a) My first improvement to the MCTS is to use minimax backups instead of
> averaging. This was an improvmenet but not something that would solve the
> traps
>
> b) My second improvment is to use alphabeta rollouts. This is a rollouts
> version that can do nullmove and LMR etc... This is a huge improvment and
> none of the MCTS
> versons can match it. More on alpha-beta rollouts here (
> https://www.microsoft.com/en-us/research/wp-content/
> uploads/2014/11/huang_rollout.pdf )
>
> So AlphaZero used none of the above improvements and yet it seems to be
> tactically strong. Leela-Zero suffered from tactical falls left and right
> too as I expected.
>
> So the only explanation left is the policy network able to avoid traps
> which I find hard to believe it can identify more than a qsearch level
> tactics.
>
> All I am saying is that my experience (as well as many others) with MCTS
> for tactical dominated games is bad, and there must be some breakthrough in
> that regard in AlphaZero
> for it to be able to compete with Stockfish on a tactical level.
>
> I am curious how Remi's attempt at Shogi using AlphaZero's method will
> turnout.
>
> regards,
> Daniel
>
>
>
>
>
>
>
>
> On Tue, Mar 6, 2018 at 9:41 AM, Brian Sheppard via Computer-go <
> computer-go@computer-go.org> wrote:
>
>> Training on Stockfish games is guaranteed to produce a blunder-fest,
>> because there are no blunders in the training set and therefore the policy
>> network never learns how to refute blunders.
>>
>>
>>
>> This is not a flaw in MCTS, but rather in the policy network. MCTS will
>> eventually search every move infinitely often, producing asymptotically
>> optimal play. But if the policy network does not provide the guidance
>> necessary to rapidly refute the blunders that occur in the search, then
>> convergence of MCTS to optimal play will be very slow.
>>
>>
>>
>> It is necessary for the network to train on self-play games using MCTS.
>> For instance, the AGZ approach samples next states during training games by
>> sampling from the distribution of visits in the search. Specifically: not
>> by choosing the most-visited play!
>>
>>
>>
>> You see how this policy trains both search and evaluation to be
>> internally consistent? The policy head is trained to refute the bad moves
>> that will come up in search, and the value head is trained to the value
>> observed by the full tree.
>>
>>
>>
>> *From:* Computer-go [mailto:computer-go-boun...@computer-go.org] *On
>> Behalf Of *Dan
>> *Sent:* Monday, March 5, 2018 4:55 AM
>> *To:*
Re: [Computer-go] 9x9 is last frontier?
Alpha-beta rollouts is like MCTS without playouts (as in AlphaZero), and
something that can also do alpha-beta pruning.
With standard MCTS, the tree converges to a minmax tree not an alpha-beta
tree, so as you know there is a huge branching factor difference there.
For MCTS to become competitive with alpha-beta, one has to use either
alpha-beta rollouts OR as in AlphaZero's case
pick accurately, say the best 3 moves with its PUCT bias, and then search
those with MCTS that would converge to minimax (not alpha-beta).
I went for the latter (recasting alphabeta in rollouts form which can be
combined with standard MCTS as well as standard recursive alpha-beta
implementation)
The paper is here
https://www.microsoft.com/en-us/research/wp-content/uploads/2014/11/huang_rollout.pdf
If it was upto me, I would have used standard alpha-beta search with
policy/value networks to improve move sorting and evaluation resp.
It seems AlphaZero's MCTS approach hinged on the policy network accurately
picking the best n moves (while still using minmax) AND then
ofcourse not making blunders. I think what "shallow traps" are some bad
looking moves turning out to be brilliant tactically later. How the policy
nework able to identify those moves ? A full width like alpha-beta does not
miss those.
Daniel
On Tue, Mar 6, 2018 at 1:23 PM, Álvaro Begué wrote:
> Sorry, I haven't been paying enough attention lately to know what
> "alpha-beta rollouts" means precisely. Can you either describe them or give
> me a reference?
>
> Thanks,
> Álvaro.
>
>
>
> On Tue, Mar 6, 2018 at 1:49 PM, Dan wrote:
>
>> I did a quick test with my MCTS chess engine wth two different
>> implementations.
>> A standard MCTS with averaging, and MCTS with alpha-beta rollouts. The
>> result is like a 600 elo difference
>>
>> Finished game 44 (scorpio-pmcts vs scorpio-mcts): 1/2-1/2 {Draw by 3-fold
>> repetition}
>> Score of scorpio-mcts vs scorpio-pmcts: 41 - 1 - 2 [0.955] 44
>> Elo difference: 528.89 +/- nan
>>
>> scorpio-mcts uses alpha-beta rollouts
>> scorpio-pmcts is "pure" mcts with averaging and UCB formula.
>>
>> Daniel
>>
>> On Tue, Mar 6, 2018 at 11:46 AM, Dan wrote:
>>
>>> I am pretty sure it is an MCTS problem and I suspect not something that
>>> could be easily solved with a policy network (could be wrong hree). My
>>> opinon is that DCNN is not
>>> a miracle worker (as somebody already mentioned here) and it is going to
>>> fail resolving tactics. I would be more than happy with it if it has same
>>> power as a qsearch to be honest.
>>>
>>> Search traps are the major problem with games like Chess, and what makes
>>> transitioning the success of DCNN from Go to Chess non trivial.
>>> The following paper discusses shallow traps that are prevalent in chess.
>>> ( https://www.aaai.org/ocs/index.php/ICAPS/ICAPS10/paper/downl
>>> oad/1458/1571 )
>>> They mention traps make MCTS very inefficient. Even if the MCTS is
>>> given 50x more time is needed by an exhaustive minimax tree, it could fail
>>> to find a level-5 or level-7 trap.
>>> It will spend, f.i, 95% of its time searching an asymetric tree of depth
>>> > 7 when a shallow trap of depth-7 exists, thus, missing to find the
>>> level-7 trap.
>>> This is very hard to solve even if you have unlimited power.
>>>
>>> The plain MCTS as used by AlphaZero is the most ill-suited MCTS version
>>> in my opinion and i have hard a hard time seeing how it can be competitive
>>> with Stockfish tactically.
>>>
>>> My MCTS chess engine with AlphaZero like MCTS was averaging was missing
>>> a lot of tactics. I don't use policy or eval networks but qsearch() for
>>> eval, and the policy is basically
>>> choosing which ever moves leads to a higher eval.
>>>
>>> a) My first improvement to the MCTS is to use minimax backups instead of
>>> averaging. This was an improvmenet but not something that would solve the
>>> traps
>>>
>>> b) My second improvment is to use alphabeta rollouts. This is a rollouts
>>> version that can do nullmove and LMR etc... This is a huge improvment and
>>> none of the MCTS
>>> versons can match it. More on alpha-beta rollouts here (
>>> https://www.microsoft.com/en-us/research/wp-content/upload
>>> s/2014/11/huang_rollout.pdf )
>>>
>>> So AlphaZero used none of the above improvements and yet it seems to be
>>> tactically strong. Leela-Zero suffered from tactical falls left and right
>>> too as I expected.
>>>
Re: [Computer-go] 9x9 is last frontier?
Hi Ingo, There is actually no randomness in the algorithm, just like AlphaZero's. It is the same algorithm as a recursive alpha-beta searcher, with the only difference being the rollouts version examines one leaf per episode (one path from root to leaf). This opens the door for mixing alpha-beta with MCTS; some have already used is an MCTS-ab algorithm that does exactly this. At the leaves, I call a qsearch() for evaluation whereas AlphaZero uses Value Network (no playouts ). Daniel On Wed, Mar 7, 2018 at 2:20 AM, "Ingo Althöfer" <3-hirn-ver...@gmx.de> wrote: > Hi Dan, > > I find your definition of "Alpha-Beta rollouts" somewhat puzzling. > > > Alpha-beta rollouts is like MCTS without playouts (as in > > AlphaZero), and something that can also do alpha-beta pruning. > > I would instead define "Alpha-Beta rollout" in the following way: > You have a fast alpha-beta program (with some randomness) for > that game, and a rollout means a quick selfplay game of this > program. > > Ingo. > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [computer-go] Re: Strength of Monte-Carlo w/ UCT...
Robert Waite skrev: * > The MCTS technique appears to be extremely scalable. The theoretical *> >* > papers about it claim that it scales up to perfect play in theory. *>* > We agree here that this is not true of course. * No, I think we disagree this time my friend! Monte Carlo of course by itself is not scalable. But when combined with tree search such as UCT, it is equivalent to a mini-max search with a high quality evaluation at leaf nodes. It's scalable because the longer it searches, the more it acts like a proper mini-max search. I am not really sure what you guys meant by the interchange above.. and I will admit that I am not nearly as deep into the problem as probably anyone on this list.. but it seems doubtful that they have proven some sort of scalable algorithm that will continue to give moves closer and closer to perfect play with higher and higher confidence. I could believe that they have a proof that the algorithm is scalable in computing power... but not necessarily that it is scalable against the problem of beating a human. MC/UCT is provably scalable up to perfect play. /Dan Andersson Firstly.. I don't know if there is necessarily "perfect" play. If you mean play good enough to beat all humans... I guess you could consider that perfect in a sense. It just seems that given the ridiculously large number of possible games... and the relative weakness our our computation machines... we will be taking an extremely small sample size with our current models of computing. There may be some breakthrough... but it seems that as long as Moore's law holds... we will continue to be taking a very small sample. There are some heuristics being used to help whittle down the bad.. like checking the 8 spots around a stone... but is it really proven that this algorithm will scale in a practical sense? As in... if we took Mogo in it's current form and threw all the hardware that humanity can make in the next 20 years at it... has that alone been shown to be enough to beat humans? Seems that we don't understand all of the parameters of human play to answer a question like that. I am playing the Devil's advocate here.. because I do have a feeling that the human computer with regards to go is not mystical and that we have a weakness that can be exploited by computers. But I just feel that there is no rigorous proof yet about the power of scalability of computation... vs. the ability to beat a human in go. On Sat, Aug 9, 2008 at 2:54 PM, Robert Waite <[EMAIL PROTECTED]> wrote: I'm curious what you guys think about the scalability of monte carlo with UCT. Let's say we took a cluster like that which was used for the Mogo vs. Kim game. Then lets say we made 128 of these clusters and connected them together efficiently. Putting aside implementation and latency issues... what kind of stones-strength increase would you imagine? Its a pretty arbitrary guess.. but do you think one stone improvement... or that this would alone be enough to beat a pro even? I am wondering because there could be a weakness or limit in MC w/ UCT. I am only now learning about the UCT addition... but there are vast numbers of possible games that are never visited during the monte carlo simulations. The random stone simulations are pretty random aren't they? I am reading some of the papers on the UCT addition... and that does seem to show certain branches to be "better" and worth more time. Pro players may have a counter-strategy that might come out as Mogo is tested at higher levels of play. Perhaps there will be a need to combine MCwUCT with heuristics or more knowledge based play. Going the route of heuristics seems unpleasant and the promise of using a more computational method would be great. However... if MC techniques alone have a diminishing return... the route of heuristics might come back (or perhaps a whole new paradigm for game algorithms). I am still secretly on the side of human go beating the machine.. but the recent match really changed my view on topic and really showed the value of statistical analysis. I am just wondering about what kind of roadblocks might show up for the monte carlo techniques. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Re: Strength of Monte-Carlo w/ UCT...
Robert Waite skrev: MC/UCT is provably scalable up to perfect play. Really? Could you send me a link to the paper? I think we must have a different definition for some word. Perfect play? Are you saying that we have proven that the 19x19 go board has a perfect path for black? I did not realize we knew so much about the problem. If it is proven to be scalable up to perfect play... then that would imply that we have some notion of how far away we are from perfect play. Or at the very least.. that we know what the graph of perfect play looks like. Maybe I am way behind in the theory.. but this seems incredible to me. No more incredible than that Mini-Max and Alpha-Beta will generate perfect play given enough resources. In the worst case MC/UCT will build a Mini-Max tree solving the game. /Dan Andersson ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] programming languages
Don Dailey skrev: I have looked at Lisaac a bit. It does seem to be pretty close to C/C++ in performance. I'm all for a prototype based OO that fits the systems programming niche. It seems heavily influences by Eiffel and Self/Smalltalk. Since it actually emits C code it's not truly faster than C :) There is however a new kid on the block that is pretty impressive and manages to beat C/C++. ATS (Applied Type System) also emits C but due to the structure and type system the code is pretty specialized and optimized to beat any sane and optimized C program. /Dan I have noticed there are 3 more languages that have appeared that seem to promise the speed and memory benefits of C and yet are higher level languages: Has anyone looked at lisaac, Vala, or Genie? Lisaac is claimed to be just as fast as C and was designed to be used to write operating systems just like C was. It looks extremely interesting but there appears to be little or no activity with it for quite a while. Vala is a C#/Java type of language that compiles to C code and also is claimed to give the advantages of C#/Java but the low level speed and control of C.You can compile stand-alone executable programs with it, unlike most other interesting languages. Genie is a brother to Vala and looks more like a static Python. The web page blurb goes like this: The syntax of Genie is designed to be clean, clear and concise. In some ways its cleaner than Python and is certainly a lot more concise than C#/Java. Genie of course has all the advantages of Vala and you can seamlessly use Vala code and bindings with it too just like a CLR. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Publishing source for (yet another) MC simulator
Don Dailey skrev: On Tue, 2008-11-04 at 09:24 -0800, Zach Tellman wrote: There are few languages other than these that offer reasonable performance, not worse than 2X slower than C, but they tend to be memory hogs. Java is one of them. I cannot imagine every seeing a top chess program written in Java, or anything that is really memory intensive as good GO program tend to be. Is any of the 2200+ GO programs on CGOS written in anything other than C? To be fair, I did write an early version of Ergo in OCaml, and it was equally fast and arguably quite a bit cleaner. I returned to C++ because an OCaml library would be useful to probably no one but myself. So if you want to release something to the public, I agree that C/C++ is the way to go. Other than that, I don't think that computer go development has been extensive enough to argue that it's spanned all possible approaches. I agree. I was intentionally a bit provocative here as I want people to comment on other languages that might qualify. I had completely forgotten about OCaml and that might very well be a possibility too. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ It's good to be a bit provocative :) Speed of execution is but one of a multitude of axes to consider. Expressiveness is another one. A more concise representation should allow for faster turnaround and testing. Reading, writing and grok time add up pretty fast unless the compiler is slow as a dog. A decent type system to catch bugs. Latest Java has a more decent than not type system. C/C++ type systems are best not spoken about. If one takes the position that bugs have a pretty significant impact on the strength of a program (A position I agree with) one could be pretty forgiving about the speed of execution of an algorithm if it is written in an expressive language with a top of the line type system since it delivers a double whammy to bugs. It actually doesn't seem like a bad move to be half or third speed to the top of the line speed monster then. Several other lines of reasoning might lead to different choices also. Interactive development and testing and/or superior visualization frameworks might lead to breakthroughs in pruning/exploration v. exploitation ratios/patterns/generalization techniques. /Dan Anderson ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Publishing source for (yet another) MC simulator
Don Dailey skrev: On Tue, 2008-11-04 at 20:13 +0100, Dan Andersson wrote: If one takes the position that bugs have a pretty significant impact on the strength of a program (A position I agree with) one could be pretty forgiving about the speed of execution of an algorithm if it is written in an expressive language with a top of the line type system since it delivers a double whammy to bugs. It actually doesn't seem like a bad move to be half or third speed to the top of the line speed monster then. It's possible to even write C code without too many bugs if you use good programming discipline, lots of assertions and so on. However, what is important when we talk about the ultimate program is the "release version" of a program. It may take a lot longer to get the bugs out, but this cost is payed for many times over by every user of your software. The reasoning about developmental time vs execution time only applies in certain situations. If you are writing a program that you only run 1 time and you don't care how long it takes to finish (within reason), then of course it strikes with a vengeance - you want the easiest system to program with.It's also cheaper to buy computers than to buy programmers, so you want something that maximizes programmers productivity.But in some cases, whatever time you spend up front developing will reward you forever. For instance if you are a commercial games developer, the end user will not appreciate that your product sucks because you chose to make it easy on yourself. One approach is to build your prototypes in whatever language really works for you, then build the final release version in something fast. I believe that if you know exactly what program you are writing - and no experimentation is involved, then it doesn't take long to build it in any language that you are familiar with. I built those reference bots very quickly, especially considering that I didn't know how to program in those languages. But I had a definite specification. I wasn't trying to figure out what to write, but I knew exactly what it was to do before I started. Several other lines of reasoning might lead to different choices also. Of course, it's all about what it is you want to maximize. Interactive development and testing and/or superior visualization frameworks might lead to breakthroughs in pruning/exploration v. exploitation ratios/patterns/generalization techniques. I agree there. You need to actually spend a lot of time up front no matter what language you are using building tools that will help you visualize, assess, instrument and evaluate versions of your software. But the language choice could very well lead you down more (or less) productive pathways. - Don Writing bug free C code aside. Deciding that it is gainful to do an "Ultimate" or release version of a program is not the easiest thing. If your project is open ended in time the time spent might never be recouped in the long run. Each premature release will set you back from the ideal maximal + final optimized release. It might not even be if you have a project with an iron clad mission statement, road map and clearly stated goals and stop date if a maximized playing strength is the primary goal. Hope the thoughts expressed are coherent. I'm tired beyond belief. /Dan Andersson ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [Computer-go] Having an "estimated winrate" on the AGA pro game broadcasts
Hi Andrew, I enjoy the AGA broadcasts. I am mostly just an observer on this list rather than a go programming expert, but I do have a couple of things to say on this topic: 1. This is done all the time on chess broadcasts these days, and I hate it. I want to know what a strong player thinks is going on, not a computer. This is a bigger "problem" with chess than go because computers are now hugely stronger than the pro commentators, so the commentators become largely reduced to interpreting the computer output rather than getting across their own human ideas. Of course, a) this is less of an issue right now as go programs (except for AlphaGo, which is private) are not quite pro strength yet, b) top-level go games have less volatile swings than chess games, and perhaps most importantly c) my opinion may be in the minority. 2. Win rate is tough. Many programs can produce the win rate of their Monte Carlo Tree Search, but because that involves looking at lots of suboptimal moves on both sides, the reported win rate is closer to 50% than it should be. A reported win rate of 60% (in the futures explored by MCTS) probably means at least 80% in reality, assuming high-level play on both sides. Crazy Stone reports an evaluation (like B+3.5) and confidence though it cautions against taking it too seriously. My assumption is that it's something like the median result, and standard deviation, of all the MCTS playouts. I find this more useful than the win rate it provides. Dan On Wed, Aug 31, 2016 at 6:41 PM, Jackson, Andrew wrote: > Hello all! Long time lurker, first time poster: I'm Andrew Jackson, i > volunteer w/ the AGA in a number of capacities. > > Most recently, i've been putting together live broadcasts of games -- > professional games from CJK, and also games from AGA events [1] > > These broadcasts feature professional commentary to help amateurs > understand what's going on, but the number one question we get in the chat > during the broadcasts is invariably "who's winning?" I was hoping > computer-go could provide some suggestions on how we might run the > game-in-progress through an engine to get an 'estimated winrate' -- or > better yet, a graph -- that could be overlaid in the corner of the screen. > > Which brings me to the computer-go mailing list :) It seems like someone > on this mailing list would probably have some good ideas about how this > might be accomplished :) What do you think? > > > Here are some more details: > > The game sgf is kept mirrored on KGS by a human transcriber w/ permission > from the KBA. > > The overlay for these broadcasts is already rendered as a local webserver > hosting a simple html shell holding the graphics; my thought was that this > server could easily make requests to another service somewhere. That > service would be tracking the game and able to send back some json > describing the engine's view of the game, its confidence, etc. We could > then plot with d3.js right on the overlay. > > The service would need to run somewhere else, as the computer streaming > the game is already using all the CPU for transcoding & pushing the video. > > Given general overview of the goal & constraints, I ask you: > > - Does this sound plausible? > - Any ideas for an engine that would be easy to hook this into? > - Are bots capable of spectating games on KGS? > > Thanks in advance for any help you might provide! > -Andrew > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] computergo.org domain
Joshua was referring to computergo.org; the email list is on computer-go.org . Dan On Tue, Sep 27, 2016 at 11:26 AM, Jim O'Flaherty wrote: > Are you implying this email list will stop functioning if this domain > isn't renewed? > > On Tue, Sep 27, 2016 at 12:07 AM, Joshua Shriver > wrote: > >> My domain expires in 6 days, so heads up it's free to grab if anyone >> wants it. >> >> -Josh >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go > > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] AlphaGo Zero
On Fri, Oct 20, 2017 at 12:06 AM, Robert Jasiek wrote: > > 3) Where is the world-wide discussion preventing a combination of AI and > (nano-)robots, which self-replicate or permanently ensure energy access, > from causing extinction of mankind? > You will find it if you Google for "artificial intelligence existential threat". But the subject seems off-topic here. Dan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] MCTS with win-draw-loss scores
The AlphaZero paper says that they just assign values 1, 0, and -1 to wins, draws, and losses respectively. This is fine for maximizing your expected value over an infinite number of games given the way that chess tournaments (to pick the example that I'm familiar with) are typically scored, where you get 1, 0.5, and 0 points respectively for wins, draws, and losses. However 1) not all tournaments use this scoring system (3/1/0 is popular these days, to discourage draws), and 2) this system doesn't account for must-win situations where a draw is as bad as a loss (say you are 1 point behind your opponent and it's the last game of a match). Ideally you'd keep track of all three probabilities and use some linear meta-scoring function on top of them. I don't think it's trivial to extend the AlphaZero architecture to handle this, though. Maybe it is sufficient to train with the standard meta-scoring (while keeping track of the separate W/D/L probabilities) but then use the currently applicable meta-scoring while playing. Your policy network won't quite match your current situation, but at least your value network and search will. On Tue, Feb 13, 2018 at 10:05 AM, "Ingo Althöfer" <3-hirn-ver...@gmx.de> wrote: > Hello, > > what is known about proper MCTS procedures for games > which do not only have wins and losses, but also draws > (like chess, Shogi or Go with integral komi)? > > Should neural nets provide (win, draw, loss)-probabilities > for positions in such games? > > Ingo. > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] MCTS with win-draw-loss scores
Do you intend to use the same draw values for both sides in the self-play games? They can be independent: - in a 3/1/0 scenario, neither player is especially happy with a draw (and in fact would rather each throw a game to each other in a two-game match than make two draws, but that's a separate issue); - in a match with one game left, both players agree that a draw and a Black win (say) are equivalent results; - in a tournament, the must-win situations of both players could be independent. In real life you usually have a good sense of how your opponent's "must-win" parameter is set, but that doesn't really apply here. On Tue, Feb 13, 2018 at 10:58 AM, David Wu wrote: > Actually this pretty much solves the whole issue right? Of course the > proof would be to actually test it out, but it seems to me a pretty > straightforward solution, not nontrivial at all. > > > On Feb 13, 2018 10:52 AM, "David Wu" wrote: > > Seems to me like you could fix that in the policy too by providing an > input feature plane that indicates the value of a draw, whether 0 as > normal, or -1 for must-win, or -1/3 for 3/1/0, or 1 for only-need-not-lose, > etc. > > Then just play games with a variety of values for this parameter in your > self-play training pipeline so the policy net gets exposed to each kind of > game. > > On Feb 13, 2018 10:40 AM, "Dan Schmidt" wrote: > > The AlphaZero paper says that they just assign values 1, 0, and -1 to > wins, draws, and losses respectively. This is fine for maximizing your > expected value over an infinite number of games given the way that chess > tournaments (to pick the example that I'm familiar with) are typically > scored, where you get 1, 0.5, and 0 points respectively for wins, draws, > and losses. > > However 1) not all tournaments use this scoring system (3/1/0 is popular > these days, to discourage draws), and 2) this system doesn't account for > must-win situations where a draw is as bad as a loss (say you are 1 point > behind your opponent and it's the last game of a match). Ideally you'd keep > track of all three probabilities and use some linear meta-scoring function > on top of them. I don't think it's trivial to extend the AlphaZero > architecture to handle this, though. Maybe it is sufficient to train with > the standard meta-scoring (while keeping track of the separate W/D/L > probabilities) but then use the currently applicable meta-scoring while > playing. Your policy network won't quite match your current situation, but > at least your value network and search will. > > On Tue, Feb 13, 2018 at 10:05 AM, "Ingo Althöfer" <3-hirn-ver...@gmx.de> > wrote: > >> Hello, >> >> what is known about proper MCTS procedures for games >> which do not only have wins and losses, but also draws >> (like chess, Shogi or Go with integral komi)? >> >> Should neural nets provide (win, draw, loss)-probabilities >> for positions in such games? >> >> Ingo. >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go > > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > > > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Paper “Complexity of Go” by Robson
On Thu, Jun 21, 2018 at 1:41 PM, Mario Xerxes Castelán Castro < marioxcc...@yandex.com> wrote: Why the misandry? In English, “he” serves for both neutral and male > gender, but “she” always excludes men. > Standards change. I could give examples of constructions that used to be considered polite but are no longer, but by definition they wouldn't be polite... I'm not going to get into an argument about usage (and I hope no one else here is either, as that is not the subject of this mailing list), so this will be my only reply on the subject, but here is one starting point if you are interested in the topic, as you seem to be: https://www.apaonline.org/page/nonsexist (the American Philosophical Association's "Guidelines for Non-Sexist Use Of Language"). Searching for "gender-fair language" on the internet will turn up plenty of other resources. Dan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] New paper by DeepMind
I believe that the dependence of C(s) (formerly c_puct) on N(s) is new. The file pseudocode.py in the supplementary download sets c_base to 19652 and c_init to 1.25. Dan On Thu, Dec 6, 2018 at 5:27 PM Rémi Coulom wrote: > Hi, > > The new alphazero paper of DeepMind about chess and shogi has been > published in Science: > > > https://deepmind.com/blog/alphazero-shedding-new-light-grand-games-chess-shogi-and-go/ > > pdf: > https://deepmind.com/documents/260/alphazero_preprint.pdf > > I tried to play "spot the difference" with their previous draft, and did > not notice any very important difference. They include shogi games, which > might be appreciated by the shogi players. It seems they still don't tell > the value of their exploration coefficient, unless I missed anything. > > Also, the AlphaZero algorithm is patented: > https://patentscope2.wipo.int/search/en/detail.jsf?docId=WO2018215665 > > Rémi > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] CGOS source on github
The primary purpose of a rating system is to predict the results of future games accurately (this is the usual axiom, at least). In a one-dimensional rating system, such as Elo, where each player's skill is represented by a single number, it is impossible to have a (non-wacky) system where A is expected to beat B in a two-player match, B is expected to beat C in a two-player match, and C is expected to beat A in a two-player match. So if the players are eccentric in that respect, a one-dimensional rating system is always going to have real problems with accurate predictions. Dan On Fri, Jan 22, 2021 at 10:54 AM uurtamo wrote: > also frankly not a problem for a rating system to handle. > > a rating system shouldn't be tweaked to handle eccentricities of its > players other than the general assumptions of how a game's result is > determined (like, does it allow for "win" and "draw" and "undetermined" or > just "win"). > > s. > > > On Fri, Jan 22, 2021 at 6:29 AM David Wu wrote: > >> On Fri, Jan 22, 2021 at 8:08 AM Rémi Coulom >> wrote: >> >>> You are right that non-determinism and bot blind spots are a source of >>> problems with Elo ratings. I add randomness to the openings, but it is >>> still difficult to avoid repeating some patterns. I have just noticed that >>> the two wins of CrazyStone-81-15po against LZ_286_e6e2_p400 were caused by >>> very similar ladders in the opening: >>> http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/73.sgf >>> http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733301.sgf >>> Such a huge blind spot in such a strong engine is likely to cause rating >>> compression. >>> Rémi >>> >> >> I agree, ladders are definitely the other most noticeable way that Elo >> model assumptions may be broken, since pure-zero bots have a hard time with >> them, and can easily cause difference(A,B) + difference(B,C) to be very >> inconsistent with difference(A,C). If some of A,B,C always handle ladders >> very well and some are blind to them, then you are right that probably no >> amount of opening randomization can smooth it out. >> >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
