[EMAIL PROTECTED] wrote:
Usually one validates scorecards with the ROC curve, Pietra Index, KS
test, etc. You may be interested in the WP 14 from BIS (www.bis.org).

Regards,

Pedro

No, the validation should be done using an absolute reliability (calibration) curve. You need to verify that at all levels of predicted risk there is agreement with the true probability of failure. An ROC curve does not do that, and I doubt the others do. A resampling-corrected loess calibration curve is a good approach as implemented in the Design package's calibrate function.

Frank


-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Maithili Shiva
Sent: Tuesday, October 07, 2008 8:22 AM
To: r-help@r-project.org
Subject: [R] How to validate model?

Hi!

I am working on scorecard model and I have arrived at the regression
equation. I have used logistic regression using R.

My question is how do I validate this model? I do have hold out sample
of 5000 customers.

Please guide me. Problem is I had never used Logistic regression earlier
neither I am used to credit scoring models.

Thanks in advance

Maithili

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



--
Frank E Harrell Jr   Professor and Chair           School of Medicine
                     Department of Biostatistics   Vanderbilt University

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Reply via email to