Hi,
one comment: Claeskens and Hjort define AIC as 2*log L - 2*p for a model
with likelihood L and p parameters; consequently, they look for models
with *maximum* AIC in model selection and averaging. This differs from
the vast majority of authors (and R), who define AIC as -2*log L + 2*p
and search for the model with *minimum* AIC. Their definition of BIC is
similarly the negative of "normal" BIC.
I would compare this to defining \pi as the base of the natural
logarithm and e as the ratio of a circle's circumference to its
diameter: of course, you can do perfectly valid mathematics with your
own definitions, but it is a recipe for confusion.
Anyone who only reads Claeskens and Hjort, fires up R and selects the
model with the maximum AIC from the candidate models is in for some
*nasty* surprises.
Worse, as far as I see, Claeskens and Hjort nowhere mention that they
are using a definition that is diametrically opposed to what is
(overwhelmingly) common, and they do not comment on this.
However, Claeskens and Hjort managed to publish a book, which I have yet
to do, so it is quite possible that there is a major flaw in my
thinking. If so, I haven't found it yet, and I would be very grateful if
somebody pointed out what I misunderstand.
Otherwise, I would be *very* careful indeed about basing my analysis
strategy on their book, although the rest of the content is very helpful
indeed - you only need to remember where to switch signs and change
"maximize" to "minimize" etc.
For AIC and BIC novices, I would recommend going with Burnham &
Anderson, which Kjetil cited below.
Best,
Stephan
Kjetil Halvorsen schrieb:
You should have a look at:
"Model Selection and
Model Averaging"
Gerda Claeskens
K.U. Leuven
Nils Lid Hjort
University of Oslo
Among other this will explain that AIC and BIC really aims at different goals.
On Mon, Jul 5, 2010 at 4:20 PM, Dennis Murphy <djmu...@gmail.com> wrote:
Hi:
On Mon, Jul 5, 2010 at 7:35 AM, LosemindL <comtech....@gmail.com> wrote:
Hi all,
Could anybody please help me understand AIC and BIC and especially why do
they make sense?
Any good text that discusses model selection in detail will have some
discussion of
AIC and BIC. Frank Harrell's book 'Regression Modeling Strategies' comes
immediately
to mind, along with Hastie, Tibshirani and Friedman (Elements of Statistical
Learning)
and Burnham and Anderson's book (Model Selection and Multi-Model Inference),
but
there are many other worthy texts that cover the topic. The gist is that AIC
and BIC
penalize the log likelihood of a model by subtracting different functions of
its number
of parameters. David's suggestion of Wikipedia is also on target.
Furthermore, I am trying to devise a new metric related to the model
selection in the financial asset management industry.
As you know the industry uses Sharpe Ratio as the main performance
benchmark, which is the annualized mean of returns divided by the
annualized
standard deviation of returns.
I didn't know, but thank you for the information. Isn't this simply a
signal-to-noise
ratio quantified on an annual basis?
In model selection, we would like to choose a model that yields the highest
Sharpe Ratio.
However, the more parameters you use, the higher Sharpe Ratio you might
potentially get, and the higher risk that your model is overfitted.
I am trying to think of a AIC or BIC version of the Sharpe Ratio that
facilitates the model selection...
You might be able to make some progress if you can express the (penalized)
log likelihood as a function of the Sharpe ratio. But if you have several
years of
data in your model and the ratio is computed annually, then isn't it a
random
variable rather than a parameter? If so, it changes the nature of the
problem, no?
(Being unfamiliar with the Sharpe ratio, I fully recognize that I may be
completely
off-base in this suggestion, but I'll put it out there anyway :)
BTW, you might find the R-sig-finance list to be a more productive resource
in
this problem than R-help due to the specialized nature of the question.
HTH,
Dennis
Anybody could you please give me some pointers?
Thanks a lot!
--
View this message in context:
http://r.789695.n4.nabble.com/Can-anybody-help-me-understand-AIC-and-BIC-and-devise-a-new-metric-tp2278448p2278448.html
Sent from the R help mailing list archive at Nabble.com.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
