The following R code using different packages gives the same results for a
simple logistic regression without regularization, but different results
with regularization. This may just be a matter of different scaling of the
regularization parameters, but if anyone familiar with these packages has
insight into why the results differ, I'd appreciate hearing about it. I'm
new to R. Thanks. (Version info below. Same results on Windows and Solaris
8, except that I haven't gotten glmnet to compile on the latter.)
Robert V (Bob) Sasseen
[1]sass...@ai.sri.com
> # Some x values (predictive var).
> x = c(-2, -2, -2, -2, -1, -1, -1, 2, 2, 2, 3, 3, 3, 3)
> # Some z values--the observed outcome.
> # Effect is that for
> # x = -2, p = 1/4;
> # x = -1, p = 1/3;
> # x = 2, p = 2/3;
> # x = 3, p = 3/4.
> z = c(0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1)
> library(Design)
> g = lrm(z ~ x)
> g$coefficients
Intercept x
-0.2224842 0.4449685
> g = lrm(z ~ x, penalty = 1)
> g$coefficients
Intercept x
-0.1620727 0.3241454
> library(glmnet)
> g = glmnet(cbind(x), cbind(1-z, z), family = "binomial", lambda = 0,
standardize = FALSE)
> coef(g)
1
-0.2224843
x 0.4449687
> g = glmnet(cbind(x), cbind(1-z, z), family = "binomial", lambda = 1, alpha
= 0, standardize = FALSE)
> coef(g)
1
-0.1098361
x 0.2196721
> library(penalized)
> fit = penalized(z ~ x)
> coefficients(fit, "all")
(Intercept) x
-0.2224843 0.4449687
> fit = penalized(z ~ x, lambda2 = 1)
> coefficients(fit, "all")
(Intercept) x
-0.2060658 0.4121315
> sessionInfo()
R version 2.9.2 (2009-08-24)
i386-pc-mingw32
locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
attached base packages:
[1] splines stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] penalized_0.9-26 glmnet_1.1-3 Matrix_0.999375-30
lattice_0.17-25
[5] Design_2.3-0 Hmisc_3.7-0 survival_2.35-4
loaded via a namespace (and not attached):
[1] cluster_1.12.0 grid_2.9.2
References
1. mailto:sass...@ai.sri.com
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
