Is there a mathematica test suite we could adapt or a standardized set of tests we could use? Maybe we could take the 100 most often used functions and make a test suite?
---- LOOK ITS A SIGNATURE CLICK IF YOU DARE--- http://www.google.com/profiles/zitterbewegung On Wed, Mar 3, 2010 at 12:04 AM, David Kirkby <david.kir...@onetel.net> wrote: > Has anyone ever considered randomised testing of Sage against Mathematica? > > As long as the result is either > > a) True or False > b) An integer > > then comparison should be very easy. As a dead simple example, > > 1) Generate a large random number n. > 2) Use is_prime(n) in Sage to determine if n is prime or composite. > 3) Use PrimeQ[n] in Mathematica to see if n is prime or composite. > 4) If Sage and Mathematica disagree, write it to a log file. > > Something a bit more complex. > > 1) Generating random equation f(x) - something that one could integrate. > 2) Generate generate random upper and lower limits, 'a' and 'b' > 3) Perform a numerical integration of f(x) between between 'a' and 'b' in Sage > 4) Perform a numerical integration of f(x) between between 'a' and 'b' > in Mathematica > 5) Compare the outputs of the Sage and Mathematica > > A floating point number, would be more difficult to compare, as one > would need to consider what is a reasonable level of difference. > > Comparing symbolic results directly would be a much more difficult > task, and probably impossible without a huge effort, since you can > often write an equation in several different ways which are equal, but > a computer program could not easily be programmed to determine if they > are equal. > > One could potentially let a computer crunch away all the time, looking > for differences. Then when they are found, a human would had to > investigate why the difference occurs. > > One could then add a trac item for "Mathematica bugs" There was once a > push for a public list of Mathematica bugs. I got involved a bit with > that, but it died a death and I became more interested in Sage. > > Some of you may know of Vladimir Bondarenko, who is a strange > character who regularly used to publish Mathematica and Maple bugs he > had found. In some discussions I've had with him, he was of the > opinion that Wolfram Research took bug reports more seriously than > Maplesoft. I've never worked out what technique he uses, but I believe > is doing some randomised testing, though it is more sophisticated that > what I'm suggesting above. > > There must be a big range of problem types where this is practical - > and a much larger range where it is not. > > You could at the same also compare the time taken to execute the > operation to find areas where Sage is much faster or slower than > Mathematica. > > Dave > > -- > To post to this group, send an email to sage-devel@googlegroups.com > To unsubscribe from this group, send an email to > sage-devel+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/sage-devel > URL: http://www.sagemath.org > -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
