I've add pages comparing LLVM-3.2 and coming GCC 4.8 on
http://vmakarov.fedorapeople.org/spec/.
The pages are accessible by links named GCC-LLVM comparison, 2013, x86
and x86-64 SPEC2000 under link named 2013. You can find these links at
the bottom of the left frame.
If you prefer email for reading the comparison, here is the copy of page
accessible by link named 2013:
Comparison of GCC and LLVM in 2013.
This year the comparison is done on coming *GCC 4.8* and *LLVM 3.2*
which was released at the very end of 2012.
As usually I am focused mostly on the compiler comparison as
*optimizing* compilers on major platform x86/x86-64. I don't consider
other aspects of the compilers as quality of debug information
(especially in optimizations modes), supported languages, standards
and extensions (e.g. OMP), supported targets and ABI, support of
just-in-time compilation etc.
This year I did the comparison using following major options
equivalent with my point of view:
o *-O0 -g, -Os, -O1, -O2, -O3, -O4* for LLVM3.2
o *-O0 -g, -Os, -O1, -O2, -O3, -Ofast -flto* for GCC4.8
I tried to decrease the number of graphs which are still too many.
Therefore I removed data for -O0 -g and -Os from the graphs but still
I post some data about these modes below. If you need exact numbers
you should look at the tables from which the graphs were generated.
I had to use -O0 for compilation of SPECInt2000 254.gap for both
compilers as LLVM3.2 can not generate correct code in any optimization
mode for this test.
Here are my conclusions from analyzing the data:
o LLVM made a regress in supported non-experimental languages which
makes a performance comparison much harder for me. Earlier LLVM was
able to use GCC frontends (although old ones) including Fortran
front-end. Now *CLANG* driver when it processes Fortran programs
just calls GCC Fortran compiler. So comparison of CLANG LLVM and
GCC on SPECFP2000 has no sense (it would be just a comparison of GCC
4.8 and version of GCC standardly used on a given machine) although
you can find such comparisons on the Internet (e.g. on phoronix.com)
Therefore I had to use *Dragonegg* (a GCC plugin which uses LLVM
backend instead of GCC backend) for generation of Fortran benchmarks
by LLVM.
Although CLANG made LLVM less dependent on GCC, *still LLVM is
heavily dependent on GCC and more generally on other GNU projects*
(GOLD, binutils etc). Industrial compilers (including Intel
compilers, SUN studio compilers, OPEN64, Pathscale) usually support
triad of languages C, C++, and Fortran. It is a pretty big
investment to implement Fortran front-end especially with
language-dependent optimizations.
o The difference between LLVM and GCC on integer benchmarks is only
about 8% for -O3 and 3-4% for 32- and 64-bit peak performance (when
LTO is used by both compilers). On floating point benchmarks, the
difference is 3% and 9% for -O3 correspondingly for 32- and 64-bit
modes and 6% and 12% for the peak performance.
To see a perspective, the performance difference between LLVM2.9 and
GCC4.7 reached 20% (on SPECFP2000 in 32- and 64-bit modes for -O3).
So *LLVM made a significant progress* with the performance point of
view since 2.9 version.
I believe such progress is achieved mostly because of a *new RA*
introduced in LLVM 3.0 and *auto-vectorization*. By the way,
although new LLVM RA is much better than the old one, I think it is
a mistake that the new RA still does not use graph-coloring based RA
which has a potential to improve performance even more
o In 2011, I used LLVM with GCC front-end and showed that a *common
opinion "LLVM is faster compiler than GCC" is a myth* when you
compare compilers in modes generating the same code quality.
It is still close to true for LLVM with CLANG front-end. For
example, in case of 32-bit SPECInt2000 the code quality generated by
GCC4.8 in -O1 mode is 16% better than one generated by LLVM3.2 in
-O1 mode and 1% better than code generated by LLVM3.2 in -O2 mode,
but GCC compiler in -O1 mode is 2% and 10% faster than LLVM3.2
correspondingly in -O1 and -O2 mode. It means that GCC -O1 is
closer to CLANG LLVM3.2 -O2 with the performance and compiler speed
point of view.
Where GCC is really slower (2.5 times) than CLANG LLVM3.2 is in LTO
mode.
o *GCC has better code size optimizations (-Os)*, GCC4.8 generates in
average 6-7% smaller code (text + data segments) of SPECInt2000 than
LLVM3.2.
o In widely used debugging mode (-O0 -g), GCC4.8 is only about 5%
slower than LLVM3.2 but generates about 16% and 13% smaller and 18%
and 10% faster SPECInt2000 code correspondingly in 32-bit and 64-bit
mode.
o Despite that LLVM supports many targets, LLVM is focused mostly on
developments two of them x86/x86-64 and ARM. I see two supporting
evidence for this thesis.
One is that dragonegg supports only the two mention targets. You
even can not benchmark SPECFP for LLVM on other targets as you can
not use LLVM to compile Fortran programs.
Another one is that the quality of code generated by LLVM
for other targets is not so good as one generated by GCC. For POWER
example (second most important server architecture), LLVM rate for
SPECInt2000 or part of SPECFP2000 (4 benchmarks on C) is about 20%
worse than for GCC.
So I would not recommend switching to LLVM for any Linux
distribution because other targets are not refined as x86/x86-64
with performance point of view (there are a lot of other aspects
besides generated code performance which make such switching
unreasonable). By the way, only LLVM-3.2 binaries for MACOS
provided by LLVM site are compiled by LLVM itself. For Linux and
FreeBSD (this project officially switched from GCC to LLVM because
of new version GNU license for GCC), the binaries are still compiled
by GCC (correspondingly by GCC 4.6 and GCC 4.2).
Still I think that GCC community should pay more attention to
improving code quality for x86/x86-64 as LLVM is catching us up.
