As a follow-up to Greg's suggested graphical presentation, it seems like the Vuong test is sometimes used to compare fits of non nested models.
Brian Brian S. Cade, PhD U. S. Geological Survey Fort Collins Science Center 2150 Centre Ave., Bldg. C Fort Collins, CO 80526-8818 email: brian_c...@usgs.gov tel: 970 226-9326 From: Greg Snow <greg.s...@imail.org> To: Boris New <new.bo...@gmail.com>, "r-help@r-project.org" <r-help@r-project.org> Date: 03/23/2011 01:14 PM Subject: Re: [R] Comparing non nested models with correlation coefficients (inspired from Lorch and Myers ) Sent by: r-help-boun...@r-project.org If you are interested in the fits, then I would just plot the fits. Plot the fitted/predicted values from the 1st model as the x-values and the fitted/predicted values from the second model as the y-values. It is best to plot on a square plotting region and use asp=1, probably also doing abline(0,1) after. This gives a nice visual of how the predictions from the models compare. You could take it a step further by taking the 2 sets of predictions and doing a Tukey mean-difference plot, or a Bland-Altman plot. I am not sure that the test you suggest would be meaningful. What null hypothesis is it testing and what does it mean if it is not rejected? (what is the power to reject?). Do you really need a formal test? If not then the plot is probably more meaningful. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 > -----Original Message----- > From: r-help-boun...@r-project.org [mailto:r-help-bounces@r- > project.org] On Behalf Of Boris New > Sent: Wednesday, March 23, 2011 7:32 AM > To: r-help@r-project.org > Subject: [R] Comparing non nested models with correlation coefficients > (inspired from Lorch and Myers ) > > Hi, > > I would like to compare two models in R with the same dependant > variable but > different predictors (two different types of frequency and reaction > times as > RT). > An editor told me to have a look at Lorch and Myers 1990. > > Lorch and Myers use the following technique: > 1) they perform regressions on individual subjects' data > 2) they extract the beta weights > 3) they run t-test on these beta weights. > > The point is that I don't want to compare the "size effect" from the > different models but how well the two models fit. So my idea was to > extract > the correlation coefficients instead of betas and doing t-tests on > these. > I checked and my correlation coefficients are normally distributed... > > Is it ok to do that? > > Best regards, > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting- > guide.html > and provide commented, minimal, self-contained, reproducible code. ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
