On Dec 20, 2007 5:39 PM, Álvaro Begué <[EMAIL PROTECTED]> wrote: > I was trying to come up with my own algorithm to maximize likelihood and I > am having a hard time getting it all in my mind. I managed to write a > working algorithm for the case of logistic regression, but it was kind of > brittle and I didn't know how to extend it to the softmax case, which is > what I wanted. > > Over lunch I thought of another way of doing it that would be very general > and easy to implement. Basically, I can compute the log-likelihood for a > particular setting of the weights, and I can compute the gradient and the > Jacobian matrix (the derivative with respect to each weight or pair of > weights, respectively). Then I can approximate the function I am minimizing > using a paraboloid computed from that data, compute the minimum of the > paraboloid and take that point to be the new setting of the weights. Has > anyone tried something like this? It seems like the natural way to do it to > me.
My knowledge of the Jacobian matrix is limited. http://en.wikipedia.org/wiki/Jacobian seems to imply that your definition of the Jacobian matrix is unusual. To fit a paraboloid, don't you need the 2nd derivative as well? I'd assume you'd fit a parabola in each direction (for each gamma) and then sum them together into one paraboloid Interestingly enough, I believe assuming a paraboloid is equivalent to assume a linear first derivative (which is what I did). Of course, I have no idea how the Jacobian comes into play, so I assume I'm missing a few things. Is it possible to share a bit more of your final method? > Anyway, I would really appreciate if other people that are trying this > kind of thing could exchange some test datasets with me to see if I get to > similar weights. I think I will make the code available for anyone to use > after I get it to be satisfactory enough. I'm not that far yet. I'll share when I have it, although I plan to try and use the same starting data set (games) as Remi. I have yet to code calculation of some of the features.
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
