Could I be agreeing with Tom? Well, sort of. If you are writing a program in the context of some on-going project, trying to improve the program that does (say) multiplication, then it is exactly relevant to compare your new program to the one you propose to replace. And if you have yet another idea, you compare your new' program, and new'' etc. That is, I think, what Bill is doing. So this benchmarking is pretty much what you want to do, though if you are too clever in choosing examples, you can make really really bad decisions.
For example, the modular polynomial GCD of Collins and Brown was PROVED to be of complexity comparable to division, and clearly of far lower complexity than (say) the next best, at the time, the subresultant PRS. Yet it was terribly terribly slow. Why? wrong model of polynomials. The complexity assumed completely dense multivariate polynomials. And if they were sparse, it didn't matter, it would make believe they were dense. So analysis AND benchmarks can be deceptive, even if not deliberate. (eventually, GCD methods that accounted for sparseness took over.) As for having some Sage-ist compare different systems, it can be pretty hopeless. If you don't know about canonical rational function representation in Maxima, you will have very slow times. (hint: any use of the function "expand" is wrong.) So Maxima might seem slow, while what the benchmark may show is that you really don't know enough about how to use Maxima. For Maple, there are substantial subtleties in timing, or at least used to be. Like you cannot time a command because that includes an extra simplification, so you have to write a little program and time that. And for Mathematica, you can mess with two (or more?) kinds of arrays, by accident. And you may be comparing machine-floats with MPFR floats with quad- precision with .... UNINTENTIONALLY. So my view is that if I have a good idea and I can make something work neatly in my system (short program, clever implementation, new algorithm, etc.) it is up to me to describe it in sufficient detail that someone could redo it in some other (similarly equipped) system and see if it represents an improvement or not. That someone would be sufficiently expert in that other system, and have enough interest in making that program neat, clever, etc, that he/she would do a good job. For something at the level of multiplying two integers, I, for one am not particularly keen on hacking. Not that it is uninteresting. Not that it is solved... There are maybe a few distinct cases of interest. These come to mind.. small integers (16 bit?) moderate (32 or sometimes 64 bit), ones that fit in a float-mantissa (52 bit), small numbers X big numbers big numbers X equally big numbers numbers X powers of 2 (shift). also of course numbers just smaller and just larger than critical points in shifting between algorithms if there are a bunch. There may be cases of alignment of data on cache-line boundaries or not. On May 3, 3:36 pm, Tom Boothby <tomas.boot...@gmail.com> wrote: > > I've always been confused about people who present benchmarks of their > > own code against more of their own code, rather than to have the > > courage to compare against other people's code. > > I think this can be useful in some contexts. It can "normalize away" > the skill of the programmer and the amount of effort put into the > code, which can make an effective comparison of two strategies / > algorithms. Also, if you consider something like MPIR, it can be > tricky for an outsider to predict which algorithm gets used to > multiply two integers -- good production code is rarely as simple as a > straightforward implementation of a single algorithm. > > -- > To post to this group, send an email to sage-devel@googlegroups.com > To unsubscribe from this group, send an email to > sage-devel+unsubscr...@googlegroups.com > For more options, visit this group athttp://groups.google.com/group/sage-devel > URL:http://www.sagemath.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
