Hi Jo, OooooK, I see your point, sorry for slowness. Yes the residuals look a bit wonky on your first glm1path plot (ft2). I’ve dug into the reason and found a bug in the code for residual.glm1path nbinom fits, where it was using the wrong parameterisation of overdispersion in computing residuals. Now fixed on github ☺. All the best David
From: Joanne Potts <joa...@theanalyticaledge.com> Sent: Wednesday, 5 December 2018 9:49 AM To: David Warton <david.war...@unsw.edu.au> Cc: r-sig-ecology@r-project.org Subject: Re: mvabund: difference between 'glm1path' and 'manyglm' Thanks David for getting back to me. I think I have followed your answer, thank you, and I get that when one specifies the theta value, all the ft3$phis are now constant for each lambda. Now I wonder if there is any value of ever specifying "negative.binomial(theta) " as I did below with ft3 (cf the ?glm1path helpfile) to improve the residuals, when using the LASSO? I guess I always thought the LASSO was a more robust way to select models but it seems the residuals of ft2 suggest otherwise. These questions are motivated for some over dispersed seal-fish data for a student in Sydney (as we've discussed off list) but I guess these questions are more of a theoretical nature. I over came my social phobia of posting on a list instead of hassling you privately(!), maybe someone else can value from this discussion too :) Thanks once again, Jo On Mon, Dec 3, 2018 at 12:10 AM David Warton <david.war...@unsw.edu.au<mailto:david.war...@unsw.edu.au>> wrote: Hi Jo, Thanks for the e-mail, always good to see statistical modelling questions on this list! In the mvabund package, you can fit trait models using different methods of estimation, method=”manyglm” will fit a GLM, “glm1path” will fit a GLM with a LASSO penalty (chosen using BIC by default but there are other options). The way we coded LASSO negative binomial regression was to update estimates of the overdispersion parameter as the slope parameters update. Because the LASSO fit gives different slope parameters, it will also have a different overdispersion parameter. It probably has a larger overdispersion parameter, because the LASSO pushes slope parameters away from the best (in-sample) fit hence there is more unexplained variation in the LASSO model. All the best David Professor David Warton School of Mathematics and Statistics, Evolution & Ecology Research Centre, Centre for Ecosystem Science UNSW Sydney NSW 2052 AUSTRALIA phone +61(2) 9385 7031 fax +61(2) 9385 7123 http://www.eco-stats.unsw.edu.au<http://www.eco-stats.unsw.edu.au/> From: Joanne Potts <joa...@theanalyticaledge.com<mailto:joa...@theanalyticaledge.com>> Sent: Friday, 30 November 2018 1:51 AM To: r-sig-ecology@r-project.org<mailto:r-sig-ecology@r-project.org> Cc: David Warton <david.war...@unsw.edu.au<mailto:david.war...@unsw.edu.au>> Subject: mvabund: difference between 'glm1path' and 'manyglm' Hi David and list, Can someone please help me understand why, when changing the 'method=manyglm' argument to 'method=glm1path' under default settings (negative binomial) the estimates of theta change in the 'trait.glm' function? I have provided example code below us the antTraits data set. And you should see the plots for ft and ft3 are similar, yet ft2 is quite different, so I think I am missing something (no doubt, probably very obvious!). Advice appreciated, thank you. Jo data(antTraits) ft=traitglm(antTraits$abund,antTraits$env,antTraits$traits,method="manyglm") ft$phi ft$theta qqnorm(residuals(ft)); abline(c(0,1),col="red") ft2=traitglm(antTraits$abund,antTraits$env,antTraits$traits,method="glm1path") mean(ft2$phis) qqnorm(residuals(ft2)); abline(c(0,1),col="red") ft3=traitglm(antTraits$abund,antTraits$env,antTraits$traits,method="glm1path", negative.binomial(theta=1.641763)) 1/mean(ft3$phis) qqnorm(residuals(ft3)); abline(c(0,1),col="red") -- Kind regards, Joanne Potts ___________________ Statistical Consultant theanalyticaledge.com<http://theanalyticaledge.com> -- Kind regards, Joanne Potts ___________________ Statistical Consultant theanalyticaledge.com<http://theanalyticaledge.com> [[alternative HTML version deleted]] _______________________________________________ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
