On 8/10/07, Martin Albrecht <[EMAIL PROTECTED]> wrote:
> > Yeah sure, I didn't mean to hijack the thread. The segfaults are
> > obviously more important than the slownesses. I think your proposal is
> > more likely to succeed if we don't get sidetracked :-)
>
> So let's open up a new thread then.
>
> To summarize: We need something to test speed regressions. Problems we face
> are at least:
> - we need tests
> - we can only compare against previous results on the same machine
> - we need to deal with some uncertainness of the results (e.g. through system
> load)
> - we need code to run the tests and compare with previous results
>
> These are just some random thoughts to get the thread rolling.
Here's a crazy idea. We decide on an extension to the doctest syntax
that states that a doctest fails if it takes over a certain amount of time
to run on a machine with at least a 2Ghz processor. For example,
sage: 2 + 2 # takes at most 5s
In doctesting, if the comment includes the phrase "takes at most [xxx]s",
then local/bin/sage-doctest will run the doctest but also time it, and if it
takes more than [xxx] seconds, it flags it as an error.
To implement the above, we could replace the above line by the
following Python equivalent lines (which gets run by Python's doctest):
>>> t = walltime()
>>> 2 + 2
>>> if FASTCPU and walltime(t) > 5: assert False, "test xxx takes
too long to run"
What do you think? This has the feel of a good but easy to use idea
to me, which other
people will poke full of holes, but with sufficient discussion might
lead to something
actually useful.
William
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/
-~----------~----~----~----~------~----~------~--~---
