On Tue, Dec 13, 2011 at 1:15 PM, Nils Bruin <nbr...@sfu.ca> wrote: > I recall reading something about that in the Python documentation and > indeed, quoting from > > http://docs.python.org/library/timeit.html > > we find: > > """ > Note > > It’s tempting to calculate mean and standard deviation from the result > vector and report these. However, this is not very useful. In a > typical case, the lowest value gives a lower bound for how fast your > machine can run the given code snippet; higher values in the result > vector are typically not caused by variability in Python’s speed, but > by other processes interfering with your timing accuracy. So the min() > of the result is probably the only number you should be interested in. > After that, you should look at the entire vector and apply common > sense rather than statistics. > """
I now remember that too. However, I take that as the sort of typical thing an engineer who doesn't really understand statistics might say. They are concerned about outliers and the data not being normally distributed. The fact is that in practice the bound got from "the lowest value gives a lower bound" itself various by quite a bit between calls to timeit. Should one just keep taking minimums? Given that processors are not deterministic and do speculative execution of instructions, etc., I'm even more dubious about the above quote. I've thrown Bill Hart in the cc, since he must have worried a lot about exactly this question when trying to make low level C/assembly code fast. -- William -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
