Hey, I just want to point out that the word "exact" has two meanings. It can mean the numerically accurate p-value as Bogdan asked in his first email, or it could mean the p-value calculated from the exact distribution of the statistic(In this case, U stat). These two are actually not related, even though they all called "exact".
Best, Jiefei On Fri, Mar 19, 2021 at 9:31 PM Spencer Graves < spencer.gra...@effectivedefense.org> wrote: > > > On 2021-3-19 12:54 AM, Bogdan Tanasa wrote: > > thanks a lot, Vivek ! in other words, assuming that we work with 1000 > data > > points, > > > > shall we use EXACT = TRUE, it uses the normal approximation, > > > > while if EXACT=FALSE (for these large samples), it does not ? > > > As David Winsemius noted, the documentation is not clear. > Consider the following: > > > set.seed(1) > x <- rnorm(100) > y <- rnorm(100, 2) > > wilcox.test(x, > y)$p.value > [1] 1.172189e-25 > wilcox.test(x, y)$p.value [1] 1.172189e-25 > > > wilcox.test(x, y, EXACT=TRUE)$p.value [1] 1.172189e-25 > wilcox.test(x, > y, EXACT=TRUE)$p.value [1] 1.172189e-25 > wilcox.test(x, y, > exact=TRUE)$p.value [1] 4.123875e-32 > wilcox.test(x, y, > exact=TRUE)$p.value [1] 4.123875e-32 > > wilcox.test(x, y, > EXACT=FALSE)$p.value [1] 1.172189e-25 > wilcox.test(x, y, > EXACT=FALSE)$p.value [1] 1.172189e-25 > wilcox.test(x, y, > exact=FALSE)$p.value [1] 1.172189e-25 > wilcox.test(x, y, > exact=FALSE)$p.value [1] 1.172189e-25 > We get two values here: > 1.172189e-25 and 4.123875e-32. The first one, I think, is the normal > approximation, which is the same as exact=FALSE. I think that with > exact=FALSE, you get a permutation distribution, though I'm not sure. > You might try looking at "wilcox_test in package coin for exact, > asymptotic and Monte Carlo conditional p-values, including in the > presence of ties" to see if it is clearer. NOTE: R is case sensitive, so > "EXACT" is a different variable from "exact". It is interpreted as an > optional argument, which is not recognized and therefore ignored in this > context. > Hope this helps. > Spencer > > > > On Thu, Mar 18, 2021 at 10:47 PM Vivek Das <vd4mm...@gmail.com> wrote: > > > >> Hi Bogdan, > >> > >> You can also get the information from the link of the Wilcox.test > function > >> page. > >> > >> “By default (if exact is not specified), an exact p-value is computed if > >> the samples contain less than 50 finite values and there are no ties. > >> Otherwise, a normal approximation is used.” > >> > >> For more: > >> > >> > https://stat.ethz.ch/R-manual/R-devel/library/stats/html/wilcox.test.html > >> > >> Hope this helps! > >> > >> Best, > >> > >> VD > >> > >> > >> On Thu, Mar 18, 2021 at 10:36 PM Bogdan Tanasa <tan...@gmail.com> > wrote: > >> > >>> Dear Peter, thanks a lot. yes, we can see a very precise p-value, and > that > >>> was the request from the journal. > >>> > >>> if I may ask another question please : what is the meaning of > "exact=TRUE" > >>> or "exact=FALSE" in wilcox.test ? > >>> > >>> i can see that the "numerically precise" p-values are different. > thanks a > >>> lot ! > >>> > >>> tst = wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE) > >>> tst$p.value > >>> [1] 8.535524e-25 > >>> > >>> tst = wilcox.test(rnorm(100), rnorm(100, 2), exact=FALSE) > >>> tst$p.value > >>> [1] 3.448211e-25 > >>> > >>> On Thu, Mar 18, 2021 at 10:15 PM Peter Langfelder < > >>> peter.langfel...@gmail.com> wrote: > >>> > >>>> I thinnk the answer is much simpler. The print method for hypothesis > >>>> tests (class htest) truncates the p-values. In the above example, > >>>> instead of using > >>>> > >>>> wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE) > >>>> > >>>> and copying the output, just print the p-value: > >>>> > >>>> tst = wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE) > >>>> tst$p.value > >>>> > >>>> [1] 2.988368e-32 > >>>> > >>>> > >>>> I think this value is what the journal asks for. > >>>> > >>>> HTH, > >>>> > >>>> Peter > >>>> > >>>> On Thu, Mar 18, 2021 at 10:05 PM Spencer Graves > >>>> <spencer.gra...@effectivedefense.org> wrote: > >>>>> I would push back on that from two perspectives: > >>>>> > >>>>> > >>>>> 1. I would study exactly what the journal said very > >>>>> carefully. If they mandated "wilcox.test", that function has an > >>>>> argument called "exact". If that's what they are asking, then using > >>>>> that argument gives the exact p-value, e.g.: > >>>>> > >>>>> > >>>>> > wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE) > >>>>> > >>>>> Wilcoxon rank sum exact test > >>>>> > >>>>> data: rnorm(100) and rnorm(100, 2) > >>>>> W = 691, p-value < 2.2e-16 > >>>>> > >>>>> > >>>>> 2. If that's NOT what they are asking, then I'm not > >>>>> convinced what they are asking makes sense: There is is no such > thing > >>>>> as an "exact p value" except to the extent that certain assumptions > >>>>> hold, and all models are wrong (but some are useful), as George Box > >>>>> famously said years ago.[1] Truth only exists in mathematics, and > >>>>> that's because it's a fiction to start with ;-) > >>>>> > >>>>> > >>>>> Hope this helps. > >>>>> Spencer Graves > >>>>> > >>>>> > >>>>> [1] > >>>>> https://en.wikipedia.org/wiki/All_models_are_wrong > >>>>> > >>>>> > >>>>> On 2021-3-18 11:12 PM, Bogdan Tanasa wrote: > >>>>>> < > >>>> > https://meta.stackexchange.com/questions/362285/about-a-p-value-2-2e-16 > >>>> > >>>>>> Dear all, > >>>>>> > >>>>>> i would appreciate having your advice on the following please : > >>>>>> > >>>>>> in R, the wilcox.test() provides "a p-value < 2.2e-16", when we > >>> compare > >>>>>> sets of 1000 genes expression (in the genomics field). > >>>>>> > >>>>>> however, the journal asks us to provide the exact p value ... > >>>>>> > >>>>>> would it be legitimate to write : "p-value = 0" ? thanks a lot, > >>>>>> > >>>>>> -- bogdan > >>>>>> > >>>>>> [[alternative HTML version deleted]] > >>>>>> > >>>>>> ______________________________________________ > >>>>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > >>>>>> https://stat.ethz.ch/mailman/listinfo/r-help > >>>>>> PLEASE do read the posting guide > >>>> http://www.R-project.org/posting-guide.html > >>>>>> and provide commented, minimal, self-contained, reproducible code. > >>>>> ______________________________________________ > >>>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > >>>>> https://stat.ethz.ch/mailman/listinfo/r-help > >>>>> PLEASE do read the posting guide > >>>> http://www.R-project.org/posting-guide.html > >>>>> and provide commented, minimal, self-contained, reproducible code. > >>> [[alternative HTML version deleted]] > >>> > >>> ______________________________________________ > >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > >>> https://stat.ethz.ch/mailman/listinfo/r-help > >>> PLEASE do read the posting guide > >>> http://www.R-project.org/posting-guide.html > >>> and provide commented, minimal, self-contained, reproducible code. > >>> > >> -- > >> ---------------------------------------------------------- > >> > >> Vivek Das, PhD > >> > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
