Re: [computer-go] Re: Scalbility study: low end

2008-01-29 Thread Heikki Levanto
On Thu, Jan 24, 2008 at 03:19:52PM +0100, Magnus Persson wrote: > > As a rule of thumb I want 300 games for each datapoint and at least > 500 if I am going to make any conclusions. Ok, I think we start to have those 500 games. To my eyes, FatMan shows a clear turn in the curve at FatMan_03. Be

Re: [computer-go] Re: Scalbility study: low end

2008-01-25 Thread Don Dailey
I think when I get some time, perhaps tonight, I will post a digest of all the games updated when the page is updated. In this way,if anyone else wants to analyze the data they can. It would be a simple text file with 1 line per game like this: FatMan_03 FatMan_11 B+Resign Mogo_02 Mogo

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Heikki Levanto
On Thu, Jan 24, 2008 at 06:07:30PM -0500, Don Dailey wrote: > FatMan is a CPU hog, I think it would be good to get a lot of data > first, and then perhaps see what happens with FatMan 14.I would not > put 15 in unless 14 showed an improvement. Fair enough! Although it is too early to say

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Hideki Kato
Don Dailey: <[EMAIL PROTECTED]>: >Hi Hideki, > >No need to stop any of the weaker games since 99% of the compute time is >consumed by the strongest half. > >Also, only the new mogo's will be scheduled to play until they catch up >- however their opponent will almost always be the stronger players.

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Hideki Kato
Oh, I intended to say that this is 68% of confidence interval and if you want 95% then ... Thank you for pointed out. -Hideki >s. > >- Original Message >From: Hideki Kato <[EMAIL PROTECTED]> >To: computer-go >Sent: Thursday, January 24, 2008 8:34:42 AM >S

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Don Dailey
I also checks what the results were when both players were within 200 ELO of each other and BOTH were at least 2400 ELO and the score is 89 - 88 in white's favor. So there doesn't seem to be any overwhelming advantage at these levels. - Don Don Dailey wrote: > Ok, what the heck. > > Not e

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Don Dailey
Ok, what the heck. Not enough data to draw any conclusions, but I broke out the white and black win percentages when both players were within specific 200 point ELO windows.I ignored all other games. rangewh bl wperc bperc - --- --- - - 1000-1199 7

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Don Dailey
FatMan is a CPU hog, I think it would be good to get a lot of data first, and then perhaps see what happens with FatMan 14.I would not put 15 in unless 14 showed an improvement. I hope to find an explanation for the sudden flattening out of FatMan, which could point to a general performan

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Alain Baeckeroot
Le jeudi 24 janvier 2008, Don Dailey a écrit : > Hi Hideki, > > No need to stop any of the weaker games since 99% of the compute time is > consumed by the strongest half. > > Also, only the new mogo's will be scheduled to play until they catch up > - however their opponent will almost always be

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Don Dailey
Yes, several comments are right on the money. You cannot trust data based on less than a few hundred games - although you can clearly see very general trends (programs tend to get stronger.) You may have noticed the scale has been expanding ever since we started. Mogo13 was one time around 20

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Don Dailey
Hi Hideki, No need to stop any of the weaker games since 99% of the compute time is consumed by the strongest half. Also, only the new mogo's will be scheduled to play until they catch up - however their opponent will almost always be the stronger players. The probability of one of them playin

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Heikki Levanto
On Thu, Jan 24, 2008 at 03:19:52PM +0100, Magnus Persson wrote: > > Nothing wrong with that, I do it myself all the time for my own tests. > But I have tricked myself at lot of time plotting curves like this. Ok, when three people all tell me that I am jumping to conclusions, I am willing to be

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Magnus Persson
Quoting Heikki Levanto <[EMAIL PROTECTED]>: Of course we can wait until we get more data. I just wanted to share my observation that the curves seem to change around the level where MC playouts tend to flatten out, and hear if anyone would have some insightful comments to that. Even with the ris

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread Heikki Levanto
On Thu, Jan 24, 2008 at 10:34:42PM +0900, Hideki Kato wrote: > The numbers of games are about 200 and their ratings' standard > deviations (right of Elo) are 70 to 100, right now. To get 95% of > reliability, you have to double them. Don't you think it's too early > to conclude any? Well, I a

Re: [computer-go] Re: Scalbility study: low end

2008-01-24 Thread steve uurtamo
24, 2008 8:34:42 AM Subject: [computer-go] Re: Scalbility study: low end Heikki, The numbers of games are about 200 and their ratings' standard deviations (right of Elo) are 70 to 100, right now. To get 95% of reliability, you have to double them. Don't you think it's too early

[computer-go] Re: Scalbility study: low end

2008-01-24 Thread Hideki Kato
Heikki, The numbers of games are about 200 and their ratings' standard deviations (right of Elo) are 70 to 100, right now. To get 95% of reliability, you have to double them. Don't you think it's too early to conclude any? -Hideki Heikki Levanto: <[EMAIL PROTECTED]>: >Everyone is looking at