Simon, that was very instructivevery special thanks to you. I already noticed that the model was bad, but it was not clear to me that transformation of predictors to, say a more centered distribution is helpful here. And thanks for pointing out Tweedie, I noticed that the error structure is far from normal and more like gamma or poisson, but Gamma made things worse.
Best regards, Jan Am 18 Apr 2013 um 17:25 schrieb Simon Wood: > Jan, > > Thanks for the data (off list). The p-value computations are based on the > approximation that things are approximately normal on the linear predictor > scale, but actually they are no where close to normal in this case, which is > why the p-values look inconsistent. The reason that the approximate normality > assumption doesn't hold is that the model is quite a poor fit. If you take a > look at gam.check(fit) you'll see that the constant variance assumption of > quasi(link=log) is violated quite badly, and the residual distribution is > really quite odd (plot residuals against fitted as well). Also see > plot(fit,pages=1,scale=0) - it shows ballooning confidence intervals and > smooth estimates that are so low in places that they might as well be minus > infinity (given log link) - clearly something is wrong with this model! > > I would be inclined to reset all the 0's to 0 (rather than 0.01), and then to > try Tweedie(p=1.5,link=log) as the family. Also the predictor variables are > very skewed which is giving leverage problems, so I would transform them to > give less skew. e.g. Something like > > fit<-gam(target~s(log(mgs))+s(I(gsd^.5))+s(I(mud^.25))+s(log(ssCmax)), > family=Tweedie(p=1.6,link=log),data=df,method="REML") > > gives a model that is closer to being reasonable (p-values are then > consistent between select=TRUE and FALSE). > > best, > Simon > > On 18/04/13 14:24, Simon Wood wrote: >> Jan, >> >> Thanks for this. Is there any chance that you could send me the data off >> list and I'll try to figure out what is happening? (Under the >> understanding that I'll only use the data for investigating this issue, >> of course). >> >> best, >> Simon >> >> on 18/04/13 11:11, Jan Holstein wrote: >>> Simon, >>> >>> thanks for the reply, I guess I'm pretty much up to date using >>> mgcv 1.7-22. >>> Upgrading to R 3.0.0 also didn't do any change. >>> >>> Unfortunately using method="REML" does not make any difference: >>> >>> ####### first with "select=FALSE" >>>> fit<-gam(target >>>> ~s(mgs)+s(gsd)+s(mud)+s(ssCmax),family=quasi(link=log),data=wspe1,method="REML",select=F) >>>> >>>> summary(fit) >>> >>> Family: quasi >>> Link function: log >>> Formula: >>> target ~ s(mgs) + s(gsd) + s(mud) + s(ssCmax) >>> Parametric coefficients: >>> Estimate Std. Error t value Pr(>|t|) >>> (Intercept) -4.724 7.462 -0.633 0.527 >>> Approximate significance of smooth terms: >>> edf Ref.df F p-value >>> s(mgs) 3.118 3.492 0.099 0.974 >>> s(gsd) 6.377 7.044 15.596 <2e-16 *** >>> s(mud) 8.837 8.971 18.832 <2e-16 *** >>> s(ssCmax) 3.886 4.051 2.342 0.052 . >>> --- >>> Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1 >>> R-sq.(adj) = 0.403 Deviance explained = 40.6% >>> REML score = 33186 Scale est. = 8.7812e+05 n = 4511 >>> >>> >>> >>> >>> >>> #### Then using "select=T" >>> >>>> fit2<-gam(target >>>> ~s(mgs)+s(gsd)+s(mud)+s(ssCmax),family=quasi(link=log),data=wspe1,method="REML",select=TRUE) >>>> >>>> summary(fit2) >>> Family: quasi >>> Link function: log >>> Formula: >>> target ~ s(mgs) + s(gsd) + s(mud) + s(ssCmax) >>> Parametric coefficients: >>> Estimate Std. Error t value Pr(>|t|) >>> (Intercept) -6.406 5.239 -1.223 0.222 >>> Approximate significance of smooth terms: >>> edf Ref.df F p-value >>> s(mgs) 2.844 8 25.43 <2e-16 *** >>> s(gsd) 6.071 9 14.50 <2e-16 *** >>> s(mud) 6.875 8 21.79 <2e-16 *** >>> s(ssCmax) 3.787 8 18.42 <2e-16 *** >>> --- >>> Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1 >>> R-sq.(adj) = 0.4 Deviance explained = 40.1% >>> REML score = 33203 Scale est. = 8.8359e+05 n = 4511 >>> >>> >>> >>> >>> >>> >>> >>> I played around with other families/link functions with no success >>> regarding >>> the "select" behaviour. >>> >>> Well, look at the structure of my data: >>> <http://r.789695.n4.nabble.com/file/n4664586/screen-capture-1.png> >>> >>> All possible predictor variables in principle look like this, and taken >>> alone, each and every is significant according to p-value (but not all >>> can >>> at the same time). >>> In theory, the target variable should be a hypersurface in 11dim space >>> with >>> lots of noise, but interaction of more than 2 vars gets costly (not to >>> think >>> of 11) and often enough (also without interaction) the solution does not >>> converge at minimal step size. If it does, results are usually not as >>> good >>> as without interaction. >>> >>> Any comment/advice on model setup is warmly welcome here. >>> >>> Since I don't want to try out all possible 2047 combinations of up to >>> eleven >>> predictor variables for each target variable, I currently see no other >>> way >>> than educated manual guessing. >>> >>> If you know another way of (semi-)automated model tunig/reduction, I >>> would >>> very much appreciate it >>> >>> best regards, >>> Jan >>> >>> >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://r.789695.n4.nabble.com/mgcv-how-select-significant-predictor-vars-when-using-gam-select-TRUE-using-automatic-optimization-tp4664510p4664586.html >>> >>> Sent from the R help mailing list archive at Nabble.com. >>> >>> ______________________________________________ >>> R-help@r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide >>> http://www.R-project.org/posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. >>> >> >> > > > -- > Simon Wood, Mathematical Science, University of Bath BA2 7AY UK > +44 (0)1225 386603 http://people.bath.ac.uk/sw283 [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
