There is another problem about these microbenchmarks as well--they often are ports of an originating C-version.
I just implemented the Rabbit stream cipher and after reading the papers I reviewed several versions online in Java and two in Go, as well as the now open-source C version. It seems that they all started with the C version and did a "c-to-Java" or "c-to-go-ization" of it. The results are not strong for the language. My code took me a day and runs at 830 MB/s vs 620 MB/s peak for any of these. Now this is not a microbenchmark exactly and I may have more experience than some, but it still shows that language-X would seem to do a job at 620 that it could do at 830 with cleaner code. That kind of difference is probably more than the difference between the upper half of the performance reports. The base code has the C-ism "... + (a<b)" where the boolean becomes 0/1 for false/true. That's a nice thing and I understood it in 1975 at BTL in Guilford Center NC. But it is not in Go or whatever else. So these port versions all have a function that takes a boolean, has an if statement, and returns 1 or 0 as appropriate. That works. But not anywhere fast enough. I reorganized the code to get rid of that. That made it much faster than any compiler mode or great language design special benefit. A core computational element of Rabbit is its custom highly non-linear G-operator. Every version I can find online does this with four 32-bit multiplies, 2 shifts, 2 adds, and an XOR.I did it with one 64-bit squaring, a shift, and an XOR. It is 20% faster on laptop/desktop/server CPUs. It is no big deal but the reason for the universal implementation was lost to all who ported without understanding and revisiting the needs of the algorithm. This is a common outcome. It makes me look at all such benchmarks and imagine such huge error bars of performance/memory/size/... that direct comparisons of those is not too useful. On the other hand, *I absolutely love such suites* because they are probably the best teacher of how programming language concepts work, help/hurt, and are expressed across languages. I've a long history in programming (Fortran II on PDP-8i, BTL CARDIAC, DG Nova, CDC 3300, ...) but learn new and helpful things whenever I see how people answer Project Euler questions or port benchmarks in languages I don't really know well. My advice is to imagine there is always somebody who could use any computer/language and double whatever you can do. Figure that in when you see that some code is 4% faster in some 20-option GCC compile mode (no frame pointer, no ...) OTOH, if Go is 20x slower than language X that is like waving a red flag at a bull; I must figure out why. So far, there has never been a meaningful reason. It is most often problem redefinition, softness in specification, and sometimes, a very clever data structure or coding technique. Rarely is it that X is just-unfixably-slow and Y is inherently fast. (There are some of those, but few, and even if, it is usually slightly unhelpful like places where Python is really fast and it's because the whole benchmark is a giant matrix SVD done by one Python call to BLAS/LINPACK/LAPACK under the hood. That is a valid result, but does not say much about Python vs anything else for actual code-in-Python.) A hidden surprise of these kinds of suites (and my coding of Rabbit) is that I'm looking at the generated code and asking, "how can this be really faster?" That made me realize a way to make Go code really faster for 32-bit integer intensive code like this, which is to make the compiler's rewrite rules and register assignments be able to express that two 32-bit variables are "roommates" in a 64-bit register. That's what I'll need to do by hand to get this over 1 GB/s and it would not be difficult. But if the compiler could do it that would make everything in this realm faster. Now I don't understand how to do that, I may not myself be able to do it, but a day of porting and benchmarking taught me what the next barrier to generated code performance may be. That was a surprise and generally a valuable one. If 100 people here have varied realizations like this, we'd really be able to make things fly. So nothing against small benchmarks and puzzle suites...just don't take the numbers to literally. Michael On Wed, Mar 6, 2019 at 4:32 PM 'Isaac Gouy' via golang-nuts < golang-nuts@googlegroups.com> wrote: > On Wednesday, March 6, 2019 at 4:03:52 PM UTC-8, Bakul Shah wrote: >> >> Thanks for an interesting read! >> >> Curious to know if you guys have any estimates on the number of lines, >> development time and number of bugs for each language implementation? I >> realize this is subjective but this comparison may be quite meaningful >> given that the authors had an existing reference implementation of a sort >> done in CL. It is not often one sees real world examples of multiple >> implementations done by a small team with the same goals. >> > > > One of the authors Pascal Costanza showed-up on programming reddit to > remedy some misconceptions. > > You could ask him in-that-discussion > <https://www.google.com/url?q=https%3A%2F%2Fwww.reddit.com%2Fr%2Fprogramming%2Fcomments%2Favsfc6%2Fperformance_comparison_of_go_c_and_java_for%2F&sa=D&sntz=1&usg=AFQjCNHo6Zv03vsBuuVj7BVZic2VgXcrRg> > or email him directly. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- *Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>* -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
