I've usually done my own calculations of entropy. Suppose p is a vector of
values (counts of outcomes, probabilities, etc.). Here's a simple function
to compute Shannon entropy:
shannon.entropy <- function(p)
{
if (min(p) < 0 || sum(p) <= 0)
return(NA)
p.norm <- p[p>0]/sum(p)
-sum(log2(p.norm)*p.norm)
}
The p > 0 qualifier is put in to avoid taking log of zero.
It gets a bit more complicated if you're trying to find things like
conditional entropy and mutual information, e.g., H(X|Y), I(X;Y), but the
above is a basic building block for anything more complex. My approach for
getting H(X|Y), etc. is to get a contingency table, then calculate
conditional entropy and mutual information from the table. You might also
consult a text such as Cover & Thomas, Elements of Information Theory.
Regards,
David
David Reinke
Senior Transportation Engineer/Economist
Dowling Associates, Inc.
180 Grand Avenue, Suite 250
Oakland, California 94612-3774
510.839.1742 x104 (voice)
510.839.0871 (fax)
www.dowlinginc.com
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of muhammad faisal
Sent: Tuesday, July 08, 2008 1:14 PM
To: R Help Cran
Subject: [R] calculation of entropy in R???
i want to calculate shannon entropy which is
H1,H2,H3....upto H7?
if there is any function or any package in
which i can find this entropy directly. do you have any information
please share this and i will be very thankful to you.
Regards,
++++++++++++++++++++++++++++++++++++++++++++++
MUHAMMAD FAISAL Department of Statistics and Decion Support system,
University of Vienna,Austria Res. # +4314796696104
Cell # +436503605156
Office # +431427738662
+++++++++++++++++++++++++++++++++++++++++++++++
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
