I was trying to come up with my own algorithm to maximize likelihood and I am having a hard time getting it all in my mind. I managed to write a working algorithm for the case of logistic regression, but it was kind of brittle and I didn't know how to extend it to the softmax case, which is what I wanted.
Over lunch I thought of another way of doing it that would be very general and easy to implement. Basically, I can compute the log-likelihood for a particular setting of the weights, and I can compute the gradient and the Jacobian matrix (the derivative with respect to each weight or pair of weights, respectively). Then I can approximate the function I am minimizing using a paraboloid computed from that data, compute the minimum of the paraboloid and take that point to be the new setting of the weights. Has anyone tried something like this? It seems like the natural way to do it to me. Anyway, I would really appreciate if other people that are trying this kind of thing could exchange some test datasets with me to see if I get to similar weights. I think I will make the code available for anyone to use after I get it to be satisfactory enough. Álvaro. On Dec 20, 2007 11:43 AM, Jason House <[EMAIL PROTECTED]> wrote: > > > On Dec 5, 2007 4:44 AM, Lars <[EMAIL PROTECTED]> wrote: > > > I have some questions concernig this paper of Remi: > > http://remi.coulom.free.fr/Amsterdam2007/MMGoPatterns.pdf > > > > @Remi: How many iterations you had used? > > > > Anyone of you have similar or other experiences with the algorithm? > > > > I seem to have more time to think than to code lately. I believe I've > derived an alternate update method. I'd be curious how quickly it > converges. > > Using Minorization-Maximization, Remi derived > new gamma = wins/sum(C/E) > > Using Newton-Raphson method, I derived > fractional change = (estimation error) * (fractional uncertainty)^2 > new gamma = gamma * (1 + fractional change) > Where > estimation error = (wins-expected wins) > fractional uncertainty = sigma/gamma > fractional change = new gamma/gamma - 1 > > Changing gamma based on both the estimation error and the uncertainty that > gamma is correct makes intuitive sense to me. Sigma matches remi's > definition. Because strengths are multiplied together, fractional > uncertainty makes more sense. I'm avoiding the difficult issue of > explaining why, but I do point people to consider two things: > 1. If gamma=gamma+sigma, what does that do to team strength? > (It multiplies the team strength by 1 + fractional uncertainty) > 2. Look at the similar definitions of fractional uncertainty > and fractional change. > > A less intuitive form is: > fractional change = (wins-expected wins) / (expected wins^2 - wins) > Using Remi's notation, expected wins = gamma*sum(C/E) > > Regardless of if someone else plays with the alternate equation, I'll be > playing with both after the holidays. I'd also be curious what thoughts > people have about this. > > To derive this, I applied the Newton-Raphson method to find where the > partial derivative of the log-evidence is zero (AKA, where log-evidence is > maximized). If L = log evidence, and *L = 1st partial derivative of L with > respect to gamma and **L = 2nd partial derivative of L with respect to > gamma, then new gamma - gamma = *L/**L. > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ >
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
