Confirmed & fixed upstream.
Thanks,
David
On 2012-02-07 18:43, Ali Tofigh wrote:
Hi,
I'm currently using the R package e1071 to train naive bayes
classifiers and came across a bug: When the posterior probabilities of
all classes are small, the result from the predict.naiveBayes function
become NaNs. This is an issue with the treatment of the
log-transformed probabilities inside the predict.naiveBayes function.
Here is an example to demonstrate the problem (you might need to
increase 'nvar' depending on your machine):
-------------------- 8< --------------------
N<- 100
nvar<- 60
varnames<- paste("v", 1:nvar, sep="")
dat<- sapply(1:nvar, function(dummy) {c(rnorm(N/2, 0, 1), rnorm(N/2, 10, 1))})
colnames(dat)<- varnames
out<- rep(c("a","b"), each=N/2)
names(dat)<- varnames
nb<- naiveBayes(x=dat, y=out)
new.dat<- t(rnorm(nvar, 5, 0.1))
colnames(new.dat)<- varnames
predict(nb, new.dat, type="raw")
-------------------- 8< --------------------
the results of the last line is usually NaNs. As for the solution:
To protect agains very small numbers, the e1071:::predict.naiveBayes
function takes the probabilities into log-space and adds instead of
multiplying probabilities. However, when calculating the posterior
probabilities of each class (when type = "raw"), the log of the
probabilities are exponentiated, which defeats the purpose of the
logspace transformation. I suggest the following change to the code:
Towards the end of the predict.naiveBayes function, you currently do:
L<- exp(L)
L / sum(L) # this is what is returned
you can instead use
sapply(L, function(lp) {1 / sum(exp(L - lp))})
the above comes from the following equality:
x / (x + y + z) = 1 / (1 + exp(log(y) - log(x)) + exp(log(z) - log(x)))
Best wishes,
/Ali Tofigh
--
Priv.-Doz. Dr. David Meyer
Department of Information Systems and Operations
WU
Wirtschaftsuniversität Wien
Vienna University of Economics and Business
Augasse 2-6, 1090 Vienna, Austria
Tel: +43-1-313-36-4393
Fax: +43-1-313-36-90-4393
HP: http://ec.wu.ac.at/~meyer
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
