Hi: John Fox's CAR book has some very nice examples of how the multinomial
likelihood is estimated computationally speaking. But you mentioned
multilevel earlier which sounds more complex ?
On Aug 2, 2009, nikolay12 <nikola...@gmail.com> wrote:
Thanks a lot. The info about computing the gradient will be helpful.
I admit that I am somewhat confused about the likelihood function itself.
It
is often said that you need to set a reference category. However, I found
two different implementations in Matlab for which setting the reference
category is not really obvious. Thus I wanted to find a single,
indisputable
implementation of the likelihood function for multinomous logistic
regression in R. Can't believe there is no such available.
Nick
Ravi Varadhan wrote:
>
> Hi,
>
> Providing the gradient function is generally a good idea in
optimization;
> however, it is not necessary. Almost all optimization routines will
> compute this using a simple finite-difference approximation, if they are
> not user-specified. If your function is very complicated, then you are
> more likely to make a mistake in computing analytic gradient, although
> many optimization routines also provide a check to see if the gradient
is
> correctly specified or not. But you can do this yourself using the
`grad'
> function in "numDeriv" package.
>
> Hope this is helpful,
> Ravi.
>
> ____________________________________________________________________
>
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology
> School of Medicine
> Johns Hopkins University
>
> Ph. (410) 502-2619
> email: [1]rvarad...@jhmi.edu
>
>
> ----- Original Message -----
> From: nikolay12 <[2]nikola...@gmail.com>
> Date: Sunday, August 2, 2009 3:04 am
> Subject: [R] Likelihood Function for Multinomial Logistic Regression and
> its partial derivatives
> To: [3]r-h...@r-project.org
>
>
>> Hi,
>>
>> I would like to apply the L-BFGS optimization algorithm to compute
>> the MLE
>> of a multilevel multinomial Logistic Regression.
>>
>> The likelihood formula for this model has as one of the summands the
>> formula
>> for computing the likelihood of an ordinary (single-level)
>> multinomial logit
>> regression. So I would basically need the R implementation for this
>> formula.
>> The L-BFGS algorithm also requires computing the partial derivatives
>> of that
>> formula in respect to all parameters. I would appreciate if you can
>> point me
>> to existing implementations that can do the above.
>>
>> Nick
>>
>> PS. The long story for the above:
>>
>> My data is as follows:
>>
>> - a vector of observed values (lenght = D) of the dependent multinomial
>> variable each element belonging to one of N levels of that variable
>>
>> - a matrix of corresponding observed values (O x P) of the independent
>> variables (P in total, most of them are binary but also a few are
>> integer-valued)
>>
>> - a vector of current estimates (or starting values) for the Beta
>> coefficients of the independent variables (length = P).
>>
>> This data is available for 4 different pools. The partially-pooled
model
>> that I want to compute has as a likelihood function a sum of several
>> elements, one being the classical likelihood function of a
>> multinomial logit
>> regression for each of the 4 pools.
>>
>> This is the same model as in Finkel and Manning "Hierarchical Bayesian
>> Domain Adaptation" (2009).
>>
>> --
>> View this message in context:
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [4]r-h...@r-project.org mailing list
>>
>> PLEASE do read the posting guide
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [5]r-h...@r-project.org mailing list
> [6]https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> [7]http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
--
View this message in context:
[8]http://www.nabble.com/Likelihood-Function-for-Multinomial-Logistic-Regr
ession-and-its-partial-derivatives-tp24772731p24781298.html
Sent from the R help mailing list archive at Nabble.com.
______________________________________________
[9]r-h...@r-project.org mailing list
[10]https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
[11]http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
References
1. mailto:rvarad...@jhmi.edu
2. mailto:nikola...@gmail.com
3. mailto:r-help@r-project.org
4. mailto:R-help@r-project.org
5. mailto:R-help@r-project.org
6. https://stat.ethz.ch/mailman/listinfo/r-help
7. http://www.R-project.org/posting-guide.html
8.
http://www.nabble.com/Likelihood-Function-for-Multinomial-Logistic-Regression-and-its-partial-derivatives-tp24772731p24781298.html
9. mailto:R-help@r-project.org
10. https://stat.ethz.ch/mailman/listinfo/r-help
11. http://www.R-project.org/posting-guide.html
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
