Dear R-Users, This list is observed by many great statisticians and non-statisticians.
I just want to add this valuable link to this great discussion. http://www.stat.duke.edu/~berger/p-values.html <http://www.stat.duke.edu/~berger/p-values.html>Thanks and Best Regards, S. On Sat, May 8, 2010 at 11:11 PM, David Winsemius <dwinsem...@comcast.net>wrote: > > On May 8, 2010, at 9:38 PM, Duncan Murdoch wrote: > > On 08/05/2010 9:14 PM, Joris Meys wrote: >> >>> On Sat, May 8, 2010 at 7:02 PM, Bak Kuss <bakk...@gmail.com> wrote: >>> >>> >>> Just wondering. >>>> >>>> The smallest the p-value, the closer to 'reality' (the more accurate) >>>> the model is supposed to (not) be (?). >>>> >>>> How realistic is it to be that (un-) real? >>>> >>>> >>>> >>> That's a common misconception. A p-value expresses no more than the >>> chance >>> of obtaining the dataset you observe, given that your null hypothesis >>> _and >>> your assumptions_ are true. >>> >> >> >> I'd say it expresses even less than that. A p-value is simply a >> transformation of the test statistic to a standard scale. In the nicer >> situations, if the null hypothesis is true, it'll have a uniform >> distribution on [0,1]. If H0 is false but the truth lies in the direction >> of the alternative hypothesis, the p-value should have a distribution that >> usually gives smaller values. So an unusually small value is a sign that H0 >> is false: you don't see values like 1e-6 from a U(0,1) distribution very >> often, but that could be a common outcome under the alternative hypothesis. >> (The not so nice situations make things a bit more complicated, because >> the p-value might have a discrete distribution, or a distribution that tends >> towards large values, or the U(0,1) null distribution might be a limiting >> approximation.) >> So to answer Bak, the answer is that yes, a well-designed statistic will >> give p-values that tend to be smaller the further the true model gets from >> the hypothesized one, i.e. smaller p-values are probably associated with >> larger departures from the null. But the p-value is not a good way to >> estimate that distance. Use a parameter estimate instead. >> > > And. Thank you for this paper. As a non-statistician I found it most > instructive: > > http://pubs.amstat.org/doi/pdfplus/10.1198/000313008X332421 > > -- > David. > > >> Duncan Murdoch >> >> >> Essentially, a p-value is as "real" as your >>> assumptions. In that way I can understand what Robert wants to say. But >>> with >>> lare enough datasets, bootstrapping or permutation tests gives often >>> about >>> the same p-value as the asymptotic approximation. At that moment, the >>> central limit theorem comes into play, which says that when the sample >>> size >>> is big enough, the mean is -close to- normally distributed. In those >>> cases, >>> the test statistic also follows the proposed distribution and your >>> p-value >>> is closer to "reality". Mind you, the "sample size" for a specific >>> statistic >>> is not always merely the number of observations, especially in more >>> advanced >>> methods. Plus, violations of other assumptions, like independence of the >>> observations, changes the picture again. >>> >>> The point is : what is reality? As Duncan said, a small p-value indicates >>> that your null hypothesis is not true. That's exactly what you look for, >>> because that is the proof the relation in your dataset you're looking at, >>> did not emerge merely by chance. You're not out to calculate the exact >>> chance. Robert is right, reporting an exact p-value of 1.23 e-7 doesn't >>> make >>> sense at all. But the rejection of your null-hypothesis is as real as >>> life. >>> >>> The trick is to test the correct null hypothesis, and that's were it most >>> often goes wrong... >>> >>> Cheers >>> Joris >>> >>> >>> bak >>>> >>>> p.s. I am no statistician >>>> >>>> [[alternative HTML version deleted]] >>>> >>>> ______________________________________________ >>>> R-help@r-project.org mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>> PLEASE do read the posting guide >>>> http://www.R-project.org/posting-guide.html >>>> and provide commented, minimal, self-contained, reproducible code. >>>> >>>> >>>> >>> >>> >>> >>> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > David Winsemius, MD > West Hartford, CT > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
