On Dec 21, 2007 8:53 AM, Rémi Coulom <[EMAIL PROTECTED]> wrote: > Hi, > > Minorization-maximization is a simple optimization method, and I agree > that it is likely that more efficient algorithms can be applied. > > Newton's method implies estimating the inverse of the Hessian matrix. > Really computing the inverse has a cost cubic in the size of the matrix, > so it is not practical with tens of thousands of parameters.
Well, you don't need the exact solution to the system of linear equations, so you can apply an iterative method instead of doing full Gaussian elimination. Well, and I wasn't thinking of having tens of thousands of parameters... How much data do you intend to use to get decent coefficients in a regression with such number of coefficients to be determined? Aren't they going to be extremely noisy? I am sure MM is a perfectly good algorithm for this purpose, but it has the serious down side that I don't understand it. :) I do understand the general idea behind it and how it works in some simple cases, but I don't know enough to adapt it to my particular needs. Nevertheless, Newton's method can be applied to each parameter one by > one. If I understand correctly, this is what Jason proposed. One step of > Newton's method on just one parameter has a computational cost very > similar to MM, and is much more efficient (it should converge in just > one or two iterations). Hmmm... That sounds similar to just considering the diagonal part of the Hessian matrix, or something of that sort. If the features are sort of orthogonal in some sense (uncorrelated?), this should be a good approximation, but I don't know how well this would apply to our particular regression problems. Remi, do you dump the training data to a simple text file and you then feed that to a program that does the regression, or does the program compute it on the fly from the database of positions? Álvaro.
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
