Thank you Detlef for doing these tests! > I want to get more people interested into this scaling, therefore I did > also some scaling tests fuego against pachi :) > > It is not as bad as oakfoam against pachi, but pachi scales a lot better > than fuego too. (attached file) To avoid additional complications I set > the number of playouts to the same value for both opponents. ELO is > again as defined in CGOS from winning rate.
I assume this is on 19x19? Yes, it is also my experience that pachi scales better than Fuego on the big board. I suspect that a big part of it is large patterns, which Fuego does not yet have. But it is also possible that something else contributes to better scaling, such as the UCT formula. I did some testing of Fuego vs pachi a few months ago. In the beginning, I did not know how to set up the pattern files of pachi correctly. I saw that this pachi version without patterns did not scale nearly as well, but I aborted the experiments when I saw that it was playing without patterns. My two conjectures are that 1. using knowledge from large patterns decreases the effective branching factor in pachi, and/or 2. patterns allow it to focus on better moves, improving the quality of the tree. I think part 2. is relatively clear. Part 1. is not clear to me. Does oakfoam have large patterns? I am currently working on adding a large pattern system to Fuego, but I just started the implementation so it will be a while. By the way, would it be possible to use the current svn Fuego instead of 1.1? It would be much more interesting for Fuego developers. Also, it is much stronger :) https://sourceforge.net/p/fuego/code/HEAD/tree/trunk/ Martin > > > I used: > Pachi version 10.00 (Satsugen) > > fuego 1.1 (does not show a more detailed version) > > with following configuration > > opponent_program2='/home/detlef/fuego-1.1/fuegomain/fuego' > opponent_settings2='uct_param_player ignore_clock 1\nuct_param_player > max_games '+str(playouts)+'\nuct_param_player resign_min_games 5000 > \nuct_param_search number_threads 8\nuct_max_memory 8000000000 > \nuct_param_player reuse_subtree 1' > > opponent_program3='/home/detlef/pachi/pachi -d 0 -t ='+str(playouts)+' > -r chinese threads=8,max_tree_size=2048,pondering=0,pass_all_alive ' > opponent_settings3='' > > taken from a CLOP like python file. > > > For oakfoam I tried to optimize a number of parameters which I thought > are relevant to scaling (progressive widening, ucb_c weighting of random > moves in playouts), but none of them was as relevant as I thought :( > > I hope I did not understand the playout number parameters wrong in pachi > and fuego. > > To me it seems there is a lot of potential in scaling, not only for > oakfoam... > > I read fuego and pachi mailing list too, if it is not of too much > interest here, we might change the mailing list:) > > Detlef > > > > Am Samstag, den 23.11.2013, 11:32 +0100 schrieb Detlef Schmicker: >> Just to let you know: >> >> I did a comparison of the playings strength vs. playouts. >> >> This time I used 4 times the oakfoam playouts for pachi >> (eg. 1000 for oakfoam 4000 for pachi) >> >> The graph shows how bad we become (in comparison) with more playouts:(. >>> From the games the first impression is, that the joseki becomes worse >> with more playouts e.g. >> >> http://www.physik.de/playouts2.pdf >> The plot is 1050 games fitted with a 5th order polynome. The borders of >> the plot are not statistical significant! >> >> Thanks for every hint :) >> >> Detlef >> >> >> Am Montag, den 18.11.2013, 22:45 +0100 schrieb Detlef Schmicker: >>> Am Montag, den 18.11.2013, 21:11 +0100 schrieb Petr Baudis: >>>> On Sun, Nov 17, 2013 at 03:11:22PM +0100, Erik van der Werf wrote: >>>>> make sure Pachi isn't doing any kind of pondering in the >>>>> background. >>>> >>>> Indeed, Pachi will ponder by default. Turn pondering off by passing >>>> >>>> pondering=0 >>>> >>>> on the commandline. >>> >>> Thanks a lot for the hint!!! From the command line documentation I >>> thought pondering is off by default.and I did not check it:( >>> >>> >>>> >>>>> pachi -d 0 -t =4000 -r chinese threads=1,max_tree_size=2048 >>>> >>>> Also, it may be worth passing pass_all_alive unless you are doing a >>>> sophisticated scoring procedure, to make sure Pachi captures all dead >>>> groups at the end of the game. >>>> >>>> P.S.: Do your results imply that on 4000 playouts/move, oakfoam is >>>> quite stronger than Pachi now? I'd love to hear more. :) How does the >>>> playout speed compare? >>> >>> Yes, we play even with 1000 against this settings. But I did not take >>> pondering into account, as I thought it is turned off. Therefore I do >>> not know if pachi really played 4000 playouts, as I thought. >>> >>> We have a little less than 1000 playouts/core/second. And my main aim is >>> to get the iPad version strong, therefore the strength with lower >>> playouts is more important to me. >>> >>> I did not optimize parameter against pachi alown, I started running clop >>> with three opponents gnugo level 10, pachi with this setting and >>> >>> /home/detlef/fuego-1.1/fuegomain/fuego >>> >>> with setting >>> uct_param_player ignore_clock 1 >>> uct_param_player max_games 3000 >>> uct_param_player resign_min_games 5000 >>> uct_max_memory 300000000 >>> >>> All 4 programs have comparable strenght than. >>> >>> Always happy to share any idea:) >>> >>> Detlef >>> >>> >>>> >>> >>> >>> _______________________________________________ >>> Computer-go mailing list >>> [email protected] >>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >>> >> >> >> _______________________________________________ >> Computer-go mailing list >> [email protected] >> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >> > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: playouts_oakfoam_fuego_pachi.pdf > Type: application/pdf > Size: 18499 bytes > Desc: not available > URL: > <http://dvandva.org/pipermail/computer-go/attachments/20131219/420ce2e5/attachment.pdf> > > ------------------------------ > > _______________________________________________ > Computer-go mailing list > [email protected] > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > End of Computer-go Digest, Vol 47, Issue 15 > ******************************************* > _______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
