Jim O'Flaherty, Jr. wrote: > Don, > > I think it is tenuous to predict, much less emphatically assert, that > just because the evidence is linear at the lower scale, it remains so > at higher scales. This is done all the time in science! Many things in science are considered facts that haven't been proven except in some empirical sense.
This theory of scalability isn't something "way out there" either. It's crystal clear that increased search depth or effort yields improved play until the point perfect play is reached. The only thing we are arguing about here is the shape of the curve. Although it seems unlikely that we will ever achieve perfect play, it doesn't make sense to require the "fat lady to sing" to know the general form of this curve. The evidence is not just linear at the lower scale as you claim either. I presented the evidence already. We now have decades of improvement to directly observe. This near-linear scale held from the point that a weak beginner could beat a chess program and has continued beyond the point that even many chess masters thought was unattainable, (even for humans.) And there still seems to be no end. Since we don't know how good a chess playing entity can ultimately be, we can't say for sure how far we are from perfect play, so it's possible we are not even half way there (whatever that means.) But it still seem ludicrous to expect the curve to suddenly flatten out, then when perfect play is almost achieved to suddenly dart to the final point in the graph. That is what you and others are predicting from zero evidence. Because if it isn't a relatively regular curve as I predict, it's going to have an unnatural shape. Of course someone is going to say, "this isn't chess, we are talking about go." I know how you guys think. If I could prove it someone would probably say, "yes, but ...." Yes, but that was with the positional superko rule. You have "no evidence" that this would hold with situational superko. Technically you would be correct, but you stretch the boundaries of credulity. Nevertheless, I did the experiment with 7x7 then 9x9 go. The shape of the 7x7 go curve was not interesting because it tended to be one sided. With 7.5 komi (I think) white usually won. It is still a profound enough game that only at fairly high levels did white always win. It was easy to see the curve suddenly flatten out when near perfect play was achieved. (I can't really be sure near perfect play was achieved, I can only guess. It's possible that some early profound moves needed to be played that even the stronger versions could not see.) I think 7x7 is "grainy" in the sense that there are probably only a very few main lines and if you stumble into them you will win even if you are relatively weak compared to your opponent. 9x9 though, was highly interesting. I showed this graph to the group. I'm pretty sure the graph reflected raw beginner play at the low end, and Dan level play at the high end (I don't think we actually have an easy way to calibrate this so I can only guess.) At the high end Lazarus (which isn't a great program but doesn't suck either) was doing an enormous amount of work - it was playing at a level way beyond what was possible on CGOS. I'm not guessing at this, I could compare Lazarus on CGOS directly because I knew about how strong it was (and I also used gnugo as a control program.) Gnugo could rarely win a game at the higher levels. (I also estimated that gnugo was a stronger program "intrinsically" that Lazarus although not scalable. Since gnugo is fixed, you can compare a version of Lazarus that takes the same amount of time to run. Gnugo beats that version of Lazarus under those conditions, and yet rarely can win a game at higher levels. Since I believe in the work/strength curve I would estimate that a scalable version of Gnugo would be superior to Lazarus at any level. Of course it's not clear how to properly scale up Gnugo, but I'm speaking theoretically here. (Not to mention that some algorithms are more scalable that others, I want to talk about that in a minute.) So I certainly didn't capture the entire range of ELO ratings and thus everyone can say I don't have an iron clad proof. Someone says, "but 19x19 is a different game altogether with different characteristics." My response is that the phenomenon seems to apply to every kind of 2 player perfect information game. It also applies to 19x19 GO at lower levels because I did a similar test with 19x19. I did this test also with a game that has a much higher branching factor than GO, a game called Arimaa. Arimaa was designed purposely to be difficult for computers. David Fotland currently has the best Arimaa program - an alpha/beta searcher with a lot of knowledge. This is the "quacks like a duck" situation. We have something here that looks like a duck, quacks like a duck, walks like a duck but you don't believe it's a duck because you didn't do a DNA test on it. I know a duck when I see it. It's possible that this is an elephant disguised a duck, but it's also possible that man didn't land on the moon and that it was a clever hoax. By the way, how do you stand on that issue? > While it is reasonable to assume, it is not certain. I see your point > that at this time, your theory about it applying to larger scales has > yet to be invalidated. However, this does not preclude your theory > being invalidated in the future. Nor does it make their intuitions > about ways others might be able to do so (and keep an open mind about > creating attempts) as superstitious. It just means they are yet to be > convinced of your position just as you are yet to be convinced of > theirs. Remember, the direct evidence used to support a theory that > the world was flat. That theory was later invalidated and replaced > with a new theory incorporating the old evidence as well as the new > evidence. The idea that the world was flat was based on emotion, not logic or evidence. Also, only SOME superstitious people believed this. It was known that the earth was globe over 3 thousand years ago. Of course you realize that some people today still believe that the earth is flat. I know of course that it was a common fallacy that the earth was flat, even in fairly recent times. But this is one of those things that get really distorted. The Columbus story is another example. There is fact and there is embellishment. The embellished story is always the one that gets told. Do I need to get into this? I think we are all fairly intelligent and we know that most of what we hear gets picked up as sound bites and get's repeated over and over, even if it's wrong. I remember when I ordered my first chess program - the advertisement had wording in it that has been repeated for decades. This wording included the claim that the program would "challenge the master" even though that was total nonsense. It's as if every program after that used "cut and paste" from the first commercial programs. You see that in almost everything and it's part of our culture. > > And you want other attempting to disprove your theory. It both > educates them on the current theory and challenges and possibly > convinces them to share holding your theory. And it also educates you > in the event they find some error in your > approach/assumptions/context/definitions or are actually able to > disprove your conclusion. And it is likely someone will eventually > disprove your theory while keeping the evidence upon which your theory > rests. This isn't much of a theory, it's a simple argument on the shape of the curve and is thus subject to interpretation. The best I can do is assert that if you plot ELO rating vs time logarithmically, you will get "almost" a straight line. It will gradually fall off so it isn't completely straight. I don't know that this is subject to some strict mathematically proof. > > I would encourage you to keep your theory (every cycle's sacred, every > cycle's great, if cycle's wasted, God gets quite irate) I cannot manage every single cycle and do them all justice, so perhaps I will sell them to science. > and work making assumptions based upon this being true. That's > efficient. I would also encourage others to challenge your theory and > work at invalidating your assumptions around low level efficiencies. > Both you, they and computer_go will be stronger because of it. > > > Jim > > > Don Dailey wrote: >> Hi Dave, >> >> You are doing it. No matter what evidence is presented, people will >> find a way to say it doesn't exist. As I mentioned earlier, the >> argument was that didn't apply to chess except for the first 4 or 5 ply >> - then when that didn't happen they expanded it to the first 6 or 7 and >> to this very day people are denying it - although they are looking more >> and more foolish in the process. >> >> We have already seen that this holds in GO, I did a massive study of it >> month ago on 9x9 boards and showed everyone this beautiful plot with >> straight lines showing the ELO per TIME curve which was essentially >> flat. >> I also remember the response. "ok, it applies to a small boards but >> 19x19 is a completely different game that bears no resemblance." >> So I must give up on this. I know if I do the plot again someone will >> say, "it only applies to depths we can currently test." "Surely it >> will flatten out next year when the new processors come." >> >> I cannot answer to those arguments when no evidence is presented to back >> it up other than superstition of disbelief or my favorite, "the >> testimony of experts in the field." I can only say that every bit >> of evidence we have backs up what I am saying. >> - Don >> >> >> Dave Dyer wrote: >> >>> I agree with your exposition of search as it applies to chess, but >>> I think there is a qualitative difference in Go. >>> >>> In chess, evaluators can see clear progress, in the form of material >>> balance and statically determined positional factors, so each >>> additional ply gives you more opportunity to see progress. >>> >>> Until Go evaluators give similarly strong and reliable signals, search >>> will be a very much weaker tool. >>> >>> _______________________________________________ >>> computer-go mailing list >>> computer-go@computer-go.org >>> http://www.computer-go.org/mailman/listinfo/computer-go/ >>> >>> >> _______________________________________________ >> computer-go mailing list >> computer-go@computer-go.org >> http://www.computer-go.org/mailman/listinfo/computer-go/ >> >> > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
