On 01/17/2012 11:55 AM, Essers, Jonah wrote:
Actually, I don't think I made myself clear and I wrote this late last
night....Sorry. More the issue is that the raw model predictions (from 0
to 1) have no inherent clinical value to them. I.e. They aren't "risk of
disease" or "risk of outcome". They are raw scores that are specific to
each model and are meant to discriminate one disease from another disease.
Trying to compare models is impossible because the NRI requires cutoff
values. The cutoffs are different for each model.
So, as I've done more reading, it appears the the IRI--Integrated
Discrimination Improvement Index--which is naïve to cutoff values--may be
more what I'm looking for. Does this make sense? I guess I just need a
sanity check.
Yes, the IRI makes sense to me.
I have been toying with the PredictABEL package and this seems to like my
data inputs just fine and relies on HMISC and ROCR, both packages I know
well.
Thanks
jonah
On 1/17/12 11:49 AM, "Kevin E. Thorpe"<kevin.tho...@utoronto.ca> wrote:
On 01/17/2012 07:16 AM, Essers, Jonah wrote:
Thanks for the reply. I think more the issue is whether it can be
applied
to cross-sectional data. This I'm not sure. This method is heavily cited
in the New England Journal of Medicine, but thus far I've only seen it
used with longitudinal data.
As I recall, the Pencina et al paper does not suggest it cannot be used
outside of longitudinal data. In fact, I don't remember them using
longitudinal data at all. So, unless I'm misunderstanding your
question, I think the function in Hmisc (whose name I always forget)
should be fine.
On 1/16/12 10:23 PM, "Kevin E. Thorpe"<kevin.tho...@utoronto.ca> wrote:
On 01/16/2012 08:10 PM, Essers, Jonah wrote:
Greetings,
I have generated several ROC curves and would like to compare the
AUCs.
The data are cross sectional and the outcomes are binary. I am testing
which of several models provide the best discrimination. Would it be
most
appropriate to report AUC with 95% CI's?
I have been looking in to the "net reclassification improvement" (see
below for reference) but thus far I can only find a version in Hmisc
package which requires survival data. Any idea what the best approach
is
for cross-sectional data?
I believe that the function in Hmisc that does this will also work on
binary data.
Thanks
Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating
the
added predictive ability of a new marker: from area under the ROC
curve
to
reclassification and beyond. Stat Med 2008;27:157-172
--
Kevin E. Thorpe
Biostatistician/Trialist, Applied Health Research Centre (AHRC)
Li Ka Shing Knowledge Institute of St. Michael's
Assistant Professor, Dalla Lana School of Public Health
University of Toronto
email: kevin.tho...@utoronto.ca Tel: 416.864.5776 Fax: 416.864.3016
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
