Frank E Harrell Jr Professor and Chairman School of Medicine
Department of Biostatistics Vanderbilt University
On Fri, 13 Aug 2010, Biau David wrote:
This is all very interesting indeed.
so I appreciate that the effect of a variable will depend on the
presence of other variables and that the effect of this variable has
a statistical meaning only in a specific model. With the
particularity of inconsistency if data arise from non normal
distribution as for the Cox model.
It's worse than that. In the Cox PH model the degree of non
proportional hazards depends on which variables are adjusted for.
I have two comments though; the first is statistical and the second more
methodological:
- first, either I am strict on the inconsistency phenomenon and, given the fact
that it is very unlikely that I even come close to
finding all relevant variables to a model, all models I may build are
necessarily biased OR I have to be less strict. How to use tests
(score, Wald, LR) to compare models when, at most, only one of them is unbiased?
- second (relevance first depend on the answer to comment above I guess), I
study the effect of age without adustment (model 1), after
adjusting for presentation variables (model 2) and after adjusting for
presentation and treatment variables (model 3). I certainly do not
pretend that I am truly looking at the effect of age and I know that what I
call age across the different models is not the same
(otherwise I would not compare it or I would hope not to find any difference).
I am merely looking the effect of something that is
measured by 'age' and that emcompasses imbalances at presentation, treatment,
and remaining unknown counfounders (first model),
imbalances in treatment and remaining unknown counfounders (second model), and
imbalances in the remaining confounders (third model). The
remaining unknown confounders potentially include a true effect of age, if such
a thing exists, and of other variables I don't know about
(which in fact are still presentation and treatment variables since there are
no other possibilities!).
As everything fits in a cohesive interpretation: older patients presents with
worse tumors and are undertreated (as shown by descriptive
statistics) AND as this is shown by the modelisation of the effect by an
erosion of the effect of 'age' from model 1 to 3, I was tempted,
with all due precautions, to conclude that the increased risk for older
patients was due in part because they present with worse tumors,
in part because the are undertreated, and in part because of something else.
Would that be overinterpreting the data? Should I even drop
the idea of building and presenting these models?
I'm not sure. But often we learn by predicting a predictor from the
other predictors.
Frank
Thanks again,
David Biau.
_________________________________________________________________________________________________________________________________________
De : Bert Gunter <gunter.ber...@gene.com>
À : Biau David <djmb...@yahoo.fr>
Cc : Frank Harrell <f.harr...@vanderbilt.edu>; r help list
<r-help@r-project.org>
Envoyé le : Ven 13 août 2010, 18h 22min 58s
Objet : Re: [R] Re : How to compare the effect of a variable across regression
models?
Just to amplify a bit on what Frank said...
Except in special circumstances (othogonal designs, say), regression
models are only "guaranteed" to produce useful predictions -- they may
not tell you anything meaningful about the relative effects of the
regressors because, as Frank said, that depends on both the study
design AND the true effects. So, for example, typically if one
removes a regressor from a model and refits, the values of the
remaining coefficients will change.
Moreover, it is difficult to understand even conceptually what "the
effect of a variable" should mean if, as is typical, it interacts with
other variables: the "effect" of the variable then depends on what the
values of the other variables are.
A very simple example of this that I used to use in teaching was the
effect of barbituates and alcohol on sleepiness. In the absence of
barbituates, the effect of a few drinks is to make you modestly
sleepy; in the presence of barbituates, the effect of a few drinks is
to make you modestly dead!
The point is that we live in a multivariate interactive world. Terms
like "the effect of a variable" derive from univariate thinking and
need to be very carefully defined to make sense in such a world -- and
then studies need to be carefully designed -- happenstance data are
rarely sufficient -- to quantify them.
.. Not what scientists like to hear, I think, but this is the reality.
Further comments and criticisms welcome, of course.
Cheers,
Bert Gunter
Genentech Nonclinical Statistics
On Fri, Aug 13, 2010 at 6:59 AM, Biau David <djmb...@yahoo.fr> wrote:
> OK,
>
> thank you very much for the answer.I will look into that. Hopefully I'll find
> smoething that will work out.
>
> Best,
>
> David Biau.
>
>
>
>
> ________________________________
> De : Frank Harrell <f.harr...@vanderbilt.edu>
>
> Cc : r help list <r-help@r-project.org>
> Envoyé le : Ven 13 août 2010, 15h 50min 18s
> Objet : Re: [R] How to compare the effect of a variable across regression
> models?
>
>
> David,
>
> In the Cox and many other regression models, the effect of a variable is
> context-dependent. There is an identifiability problem in what you are doing,
> as discussed by
>
> @ARTICLE{for95mod,
> author = {Ford, Ian and Norrie, John and Ahmadi, Susan},
> year = 1995,
> title = {Model inconsistency, illustrated by the {Cox} proportional hazards
> model},
> journal = Stat in Med,
> volume = 14,
> pages = {735-746},
> annote = {covariable adjustment; adjusted estimates; baseline imbalances;
> RCT; model misspecification; model identification}
> }
>
> One possible remedy, which may not work for your goals, is to embed all models
> in a grand model that is used for inference.
>
> When coefficients ARE comparable in some sense, you can use the bootstrap to
get
> confidence bands for differences in regressor effects between models.
>
> Frank
>
> Frank E Harrell Jr Professor and Chairman School of Medicine
> Department of Biostatistics Vanderbilt University
>
> On Fri, 13 Aug 2010, Biau David wrote:
>
>> Hello,
>>
>> I would like, if it is possible, to compare the effect of a variable across
>> regression models. I have looked around but I haven't found anything. Maybe
>> someone could help? Here is the problem:
>>
>> I am studying the effect of a variable (age) on an outcome (local recurrence:
>> lr). I have built 3 models:
>> - model 1: lr ~ age y = \beta_(a1).age
>> - model 2: lr ~ age + presentation variables (X_p) y = \beta_(a2).age
> +
>> \BETA_(p2).X_p
>> - model 3: lr ~ age + presentation variables + treatment variables( X_t)
>> y = \beta_(a3).age + \BETA_(p3).X_(p) + \BETA_(t3).X_t
>>
>> Presentation variables include variables such as tumor grade, tumor size,
>>etc...
>> the physician cannot interfer with these variables.
>> Treatment variables include variables such as chemotherapy, radiation,
> surgical
>> margins (a surrogate for adequate surgery).
>>
>> I have used cph for the models and restricted cubic splines (Design library)
>>for
>> age. I have noted that the effect of age decreases from model 1 to 3.
>>
>> I would like to compare the effect of age on the outcome across the different
>> models. A test of \beta_(a1) = \beta_(a2) = \beta_(a3) and then two by two
>> comparisons or a global trend test maybe? Is that possible?
>>
>> Thank you for your help,
>>
>>
>> David Biau.
>>
>>
>>
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
>
>
> [[alternative HTML version deleted]]
>
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
