Hi, François Perrad wrote: > I'm interested by the source of yours benchmarks, for me, it's > more test case.
These are the Lua benchmarks I've written for the shootout. You're already using these in your testcases. :-) SciMark for Lua is at: http://luajit.org/download/scimark-2007-09-19.lua Joshua Isom wrote: > For x86, you can also combine different runcores. If you try -Cj it might > run even faster. Well, the differences are minimal, so I've only tested -j vs. the default options (with higher runtimes). > Also, for some programs, keeping gc on can be much > faster. The --no-gc is to deal with(hopefully fixed) bugs in parrot. Unfortunately lua.pbc fails immediately without this option. I'll be happy to rerun the benchmarks whenever that problem is fixed. chromatic wrote: > JIT's much better for long-running processes. The last time I > profiled a test with JIT, the process spent most of its time > *generating* the JIT code. The result ran faster than even the > CGP code, but the process didn't run the JITted code long > enough to overcome the cost of JITting. Ok, I reran all benchmarks with at least 10 seconds runtime for Parrot. This should be enough for a meaningful test. I mean ... LuaJIT is able to JIT-compile these trivial benchmarks in less than a millisecond (yes, no typo). So here's the complete benchmark table. Note that this is only intended for comparing Parrot with Lua/LuaJIT. It's not so useful for comparing Lua with LuaJIT because of the short runtimes. I had to scale them down that much to get them to run with Parrot. Scale | Runtimes in s (LOWER IS BETTER) | factor | Lua LuaJIT | Parrot 0.4.16 | Parrot slower Benchmark N | 5.1.2 1.1.3 | default -j | than LuaJIT ---------------------|----------------|----------------|-------------- binarytrees 11 | 0.372 0.172 | 9.86 9.88 | 57x chameneos 40000 | 0.087 0.024 | 11.14 11.11 | 462x fannkuch 9 | 1.175 0.161 | 38.66 35.48 | 220x knucleotide 50000 | 0.786 0.479 | 51.19 53.61 | 111x mandelbrot 300 | 0.285 0.048 | 10.63 10.35 | 215x nbody 30000 | 0.366 0.082 | 12.49 12.20 | 148x nsieve 7 | 0.683 0.283 | 13.58 12.64 | 44x nsievebits 4 | 0.229 0.041 | 13.76 13.14 | 320x partialsums 300000 | 0.356 0.116 | 12.92 12.74 | 109x recursive 2 | 0.227 0.030 | 10.67 10.44 | 348x revcomp 100000 | 0.136 0.072 | 11.13 10.85 | 150x spectralnorm 200 | 0.443 0.107 | 16.78 16.42 | 153x sumcol 1000 | 0.453 0.416 | 10.02 10.13 | 24x | Performance (HIGHER IS BETTER) | | Lua LuaJIT | Parrot 0.4.16 | Parrot slower SciMark Sizes | 5.1.2 1.1.3 | default -j | than LuaJIT ---------------------|----------------|----------------|-------------- FFT 1024 | 7.31 48.45 | 0.29 0.31 | 156x SOR 100 | 18.64 117.50 | 1.44 1.58 | 74x MC | 4.44 23.14 | 0.14 0.16 | 145x SPARSE 1000, 5000 | 10.45 65.80 | 0.60 0.66 | 100x LU 100 | 16.41 82.45 | (failed) | - The following benchmarks failed with Lua on Parrot: fasta Error: attempt to call a nil value message Error: maximum recursion depth exceeded pidigits Parser error regexdna Wrong result SciMark LU Error: attempt to index a nil value Bye, Mike
