Sorry, I forgot that pdf file is not permitted. Therefore I am
resending my email without it.
On 1/23/2014, 5:56 PM, Chris Lattner wrote:
On Jan 23, 2014, at 12:14 PM, Steven Bosscher <stevenb....@gmail.com> wrote:
(Hint: read http://vmakarov.fedorapeople.org/spec/ as an example of a
better-supported point of view.)
Unrelated to this thread, it would be great for this web page to get updated. You may
find it to be "a better-supported point of view", but it is also comparing
against clang 3.2, which is from the end of 2012, and a lot has changed since then.
Modern clang versions have its autovectorizor on by default (like GCC's) and generate
much better code in general. http://www.phoronix.com/ has done some benchmarks across a
wider range of code than just spec (which is notoriously "hacked" by compiler
developers) and Clang generates better code (and faster) than GCC in many cases.
-Chris
I am going to do this when gcc4.9 is released. I have data for gcc4.8
and 3.3 but I never published them. Here is an excerpt from my slides
about the compiler release comparison:
------------------------------
SPEC2000 Setup
o Intel Haswell 3.4Ghz (i5-4670)
p Common opts: -mtune=corei7 -march=i686
o peak:
o LLVM: -O3
o GCC: -Ofast -fno-fast-math
o LTO:
o LLVM: -O4
o GCC: -Ofast -fno-fast-math -flto -fwhole-program
o 32-bit: additionally -mpc64
o Only 4 SPECFP2000 (C) tests as LLVM has no Fortran FE
SPEC2000 Results
SPECInt LLVM-3.2 LLVM-3.3 GCC-4.7 GCC-4.8
32-bit peak
3302 (0.0%) 3305 (+0.06%) 3525 (+6.7%) 3543 (+7.3%)
32-bit LTO
3592 (0.0%) 3596 (+0.06%) 3704 (+3.1%) 3740 (+4.2%)
64-bit peak
3537 (0.0%) 3517 (-0.57%) 3692 (+4.4%) 3724 (+5.4%)
64-bit LTO
3787 (0.0%) 3789 (+0.05%) 3806 (+0.5%) 3846 (+1.6%)
SPECFP LLVM-3.2 LLVM-3.3 GCC-4.7 GCC-4.8
32-bit peak
3583 (0.0%) 3580 (-0.08%) 4888 (+36%) 4985 (+39%)
32-bit LTO
3661 (0.0%) 3668 (+0.19%) 4878 (+33%) 5011 (+37%)
64-bit peak
5468 (0.0%) 5431 (-0.68%) 5799 (+6%) 5917 (+8.2%)
64-bit LTO
5659 (0.0%) 5659 (0.0%) 5881 (+3.9%) 6005 (+6.1%)
Conclusions
o LLVM 3.2 -> 3.3: No progress, even small 64-bit peak degradation (0.6%)
o GCC4.7-> 4.8: Steady progress (1%-4%)
o Still LLVM is dangerously close to GCC on some tests (1.6% on 64-bit
LTO SPECInt)
----------------------------------------
I also already have some comparison data for the trunk and llvm 3.4 but
I don't want to publish it here as the trunk is not a release.
As for Phoronix, so far I saw several pitfalls in their testing methodology:
o Micro-benchmarking. E.g. favorite benchmark Scimark2 contains a few
tests with only one small hot loop, like LU-factorization where most
benchmark time is spent in 2-lines loop. It means that the worse
results for GCC can be easily fixed as Jakub Jelinek recently improved
Scimark SOR by 42% by a small patch:
http://gcc.gnu.org/ml/gcc-patches/2013-12/msg01986.html
o Comparing LLVM and GCC on Fortran benchmarks. LLVM has no fortran FE
and just quietly call system GCC. So comparison of LLVM and GCC on
Fortran benchmarks means comparison of system GCC and a given GCC.
o IMHO, the data in articles lack credability may be because a wrong
setup (by me or by phoronix). E.g. I tried to reproduce Scimark results
for GCC4.8 and LLVM3.3 from his article "LLVM Clang 3.4 Already Has Some
Performance Changes":
http://www.phoronix.com/scan.php?page=article&item=llvm_clang34_first&num=2
Phoronix used i7-4770K for this. I used the closest machine I found
i5-4670 (with switched turbo mode off). The important difference is
0.1Ghz in frequency (3.5Ghz vs. 3.4 Ghz). I got GCC Scimark (-large)
composite score close to the article when I used -O and still on my
machine the composite score was 20% higher than the article reports
although the article says that -O3 -march=core-avx were used.
o Phoronix articles about LLVM and GCC usually contains a lot of
negative emotions about GCC and positive ones about LLVM. Such bias to
LLVM is suspicious at least for me and make me feel Phoronix as just
LLVM marketing machine.
Still I like that LLVM did a good progress in generated code
performance, it makes GCC people working on optimizations (including me)
to justify the importance of their work. In overall, competition is a
good thing for LLVM and GCC as it stimulates compiler developing in
faster pace. One time it was GCC vs Open64. Next time it would be GCC
and something else or LLVM and something else. Who knows.
