Weston Markham wrote: > On Tue, Apr 22, 2008 at 4:23 PM, Don Dailey <[EMAIL PROTECTED]> wrote: > >> Here is what I'm going to do: >> >> I will take an open source chess program, Toga, and run a multi-round >> robin between 7 versions from fixed depth 1 to fixed depth 7. Two >> versions of Toga at these 7 levels where one version has pawn structure, >> king safety, and passed pawns turned off. >> > ... > > I am not familiar with chess programming, and I haven't been paying > complete attention to this discussion, but I thought that I should > comment on this. Without any background knowledge, I would expect > that the bits of "knowledge" that you are turning off are present in > the starting program largely because they do scale well. > Not likely. Programmers put knowledge into chess program specifically to make them play stronger. I don't know of any programmer who tries to hand tailor knowledge like this specifically to make is scale better.
> Furthermore, if your claim is: > > "a chess program with a better evaluation function improves MORE with > increasing depth than one with a lesser evaluation function" > > ...then I don't see how you will make much progress at Settling the > Matter with this study, since all it will show (at best) is that there > exists one pair of evaluation functions that match your rule. > My plan is to do the study, document if fully so that it can be verified by anyone who wants to, and let people draw their own conclusions. If they don't agree with MY conclusions they can come to their own and perhaps back it up with their own study. > A better approach, to my mind, would be to test a wide variety of > different evaluation functions. As I understand it, you want to show > that there is a strong correlation between their relative playing > ability at (widely) different depths. Ideally, you should include as > many evaluation functions as you can manage, and ones that are as > different from each other as possible. Also, you possibly might want > to also combine them with multiple, different kinds of > pruning/searching/whatever-else-goes-into-a-chess-engine-that-isn't-considered-evaluation. > This would show that you are exposing the general rule, rather than > just an example of that rule. Am I misunderstanding your claim? > I am satisfied if I can show that it's possible to create a scalable evaluation function or in your terminology an example of the rule. If my very first test shows the effect, it would be pretty unreasonable to conclude that I just happened to get lucky. A much more reasonable conclusion would be that it's a common feature of many good evaluation terms and that it's not difficult to produce such functions. Your original assertion that all I'm proving is that this pair of functions exhibits this effect, may be technically true, but not reasonable in my opinion. Using different pruning and searching rules is not necessary. It's indisputable that different search heuristics scale better, there is nothing to prove. > Of course, that would be quite a bit of work, that I am suggesting. > Yes, and I'm not really into this that much, I'm only trying to prove something (empirically) that most chess programmers already know anyway but was never documented to my knowledge. > Perhaps a modest step in this direction would be to run tournaments > between 3 versions of Toga, each with only one enabled feature out of > the three that you identify. (Or perhaps two. However, avoid > including combinations where one version has features that are a > subset of another. This may help to mitigate objections such as my > initial one above.) On the plus side, though, I see no reason to run > any depths other than 1 and 7, since I think that you just want the > rank correlation between two different depths. > Yes, I considered only running these 2 levels but I'm also interested in the general curve and I think it's more robust to test against some opponent weaker and stronger and not just one. - Don > Weston > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
