Since my last message, I personally have only submitted a new faster Clojure program for the knucleotide problem on the Benchmarks Game site. I haven't checked whether other Clojure programs have been submitted in that time.
For all of the benchmark machines below, I'd say don't worry about the pidigits problem. It is kind of an odd case. The Clojure program does not use the GNU gmp library for arbitrary precision integer arithmetic, but the Java program and several other languages do. You can stress out about that fact if you want. I don't. Life is too short. That problem simply shows the difference between Java BigInteger performance versus the GNU gmp library performance, not Clojure vs. Java. The rest of the problems I consider reasonably fair comparisons, although I'd bet money the Clojure run time of several can still be reduced with enough skill and persistence. Some day I'd like to have side by side performance comparisons of these programs versus the easier-to-write kind that someone is more likely to come up with on a first or second try, rather than the tweaked-out oddities of programs on the site. In a few months, perhaps. 32-bit 1 core Clojure vs. Java: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=clojure&lang2=java Only 2 problems (other than pidigits) have Clojure worse than 4x the Java time. I don't know why reverse-complement and fasta-redux are so bad relative to the Java programs, because the exact same programs are at most 4x slower on the 64-bit benchmark machines. Something weird performance-wise about those programs on 32-bit machines. 32-bit 4 core: http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=clojure Only 3 problems (other than pidigits) worse than 4x the Java time. reverse-complement and fasta-redux have the same weird 32-bit performance weirdness as mentioned above, whatever that may be. fannkuch-redux is 5x the Java run time. 64-bit 1 core: http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=clojure 64-bit 4 core: http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=clojure All Clojure programs within 4x the run time of the corresponding Java programs, averaging around 2.5x the run time of Java. Lesson I've learned: If you've got an inner loop of your program that you really really need to run faster, code it in Java, C, or even assembler. Use profiling tools to find those spots, rather than guessing where you think they might be. Hand-coded assembler can beat optimized C by a factor of 2 or more in many cases, but who wants to program in assembler? For that matter, who wants to write large programs in C? Andy On Aug 28, 2012, at 7:02 AM, Ben Mabey wrote: > Thanks Andy for the insightful report! I knew you and others have worked > hard on the benchmarks so this kind of analysis is very helpful. > > Thanks for all your work on them, > Ben > > On 8/28/12 12:07 AM, Andy Fingerhut wrote: >> I've written several of the Clojure programs on the site. I'm not >> omniscient when it comes to writing efficient Clojure code, but I know a few >> of the techniques. Several, if not most, of the Clojure solutions already >> take advantage of mutable data structures, for example. >> >> There are some faster programs people have created for a few of the >> problems, but they haven't been submitted yet, due to lack of time/interest. >> I might get some of them submitted over the next several weeks. >> >> I suspect the average ratio of <Clojure run time> / <Java run time> can be >> brought closer to the range of x1 through x5 or so, rather than the current >> range of x2 through x12. It used to be in the range of x1 to x8 or so, but >> over the last year the Java program submissions have gotten faster while the >> Clojure ones haven't been improved, or not as much. Getting x1 or even x2 >> across all problems seems highly unlikely to me, but I'd be happy to be >> proved wrong. >> >> It is true that a few of the problems have very short run times for the Java >> programs that extra Clojure init time makes it difficult for them to compete >> (e.g. best Java run times for reverse-complement and fasta-redux problems >> are under 2 sec), but half of them take over 20 sec to complete using the >> Java programs, which should give both Java and Clojure programs plenty of >> time for JIT optimizations to take effect. The remaining differences are >> either differences in the algorithm or data structure in use, and/or the >> efficiency of the resulting byte codes from Java and Clojure compilers. >> >> As for memory used, "Hello, world" takes about 60 to 70 Mbytes of resident >> memory to complete, so no Clojure program will be below that. If you want >> thousands of separate JVMs running Clojure processes, each with a small >> memory footprint, I wouldn't recommend it, but I doubt anyone would want to >> do that. It is more fair to compare memory use for problems that require a >> larger amount of memory, e.g. reverse-complement, k-nucleotide (for which >> the Clojure memory footprint can probably be reduced without a lot of >> effort), regex-dna, and binary-trees problems. >> >> Andy >> >> >> On Aug 27, 2012, at 6:58 PM, Ben Mabey wrote: >> >>> Looking at clojure's benchmarks they seem to already be highly optimized >>> (in terms of employing all the standard tricks). Does anyone have any idea >>> if more could be done to lessen the gap between java and clojure[1]? Or >>> are these benchmarks representative of the performance gap between clojure >>> and java for these types of problems? >>> >>> -Ben >>> >>> >>> >>> 1. >>> http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=clojure&lang2=java >>> >>> On 8/26/12 8:07 AM, David Nolen wrote: >>>> I haven't found this to be the case. Java fares pretty well on Alioth. >>>> >>>> On Saturday, August 25, 2012, Stuart Sierra wrote: >>>> The Alioth benchmarks are somewhat unfair to JVM languages because they >>>> include startup time for the JVM itself and often don't run enough >>>> iterations to engage the optimizer. >>>> -S >>>> >>>> On Sat, Aug 25, 2012 at 1:51 PM, Raymond de Lacaze <del...@hotmail.com> >>>> wrote: >>>> > Here’s a performance benchmark comparison of SBCL and Clojure. >>>> > http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=clojure&lang2=sbcl >>>> > >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clojure@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+unsubscr...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clojure@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+unsubscr...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clojure@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+unsubscr...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with your >> first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en > > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
