Why is lto/whole program mode not used in LLVM for peak performance comparison? (of course, peak performance should really use FDO..)
thanks, David On Wed, Sep 7, 2011 at 8:15 AM, Vladimir Makarov <vmaka...@redhat.com> wrote: > Some people asked me to do comparison of GCC-4.6 and LLVM-2.9 (both > released this spring) as I did GCC-LLVM comparison in previous year. > > You can find it on http://vmakarov.fedorapeople.org/spec under > 2011 GCC-LLVM comparison tab entry. > > > This year the comparison is done on GCC 4.6 and LLVM 2.9 which were > released in spring 2011. > > As usually I am focused mostly on the compiler comparison > as *optimizing* compilers on major platform x86/x86-64. I don't > consider other aspects of the compilers as quality of debug > information, supported languages, standards and extensions (e.g. OMP), > supported targets and ABI, support of just-in-time compilation etc. > > Different to the 2010 comparison, the SPEC2000 benchmarks were run on > a recent *Sandy Bridge processor* which will be a mainstream > processor at least for the next year. > > This year I tried to decrease the number of graphs which are still too > many with my point of view. Some graphs are bigger than for 2010 > comparison and oriented to screens with a larger resolution. If you > need exact numbers you should look at the tables from which the graphs > were generated. > > I added GCC run with -O1 which helps to understand > that *LLVM with -O2 or -O3 is analog of GCC 4.1 with -O1 > with the point of view of generated code performance and > compilation speed*. People are frequently saying that LLVM is a much > faster compiler than GCC. That is probably not true. If you need the same > generated code quality and compilation speed as LLVM -O2/-O3 > you should use GCC with -O1. If you want 10%-40% faster > generated code, you should use GCC with -O2/-O3 and you need > 20%-40% (150%-200% if you use GCC LTO) more time for compilation. I > believe that LLVM code performance is far away from GCC because > it is sufficiently easy to get first percents of code improvement, it > becomes much harder to get subsequent percents, and IMHO starting with > some point of the development the relation of the code improvement to > the spent efforts might become exponential. So there is no magic -- > GCC has a better performance because much more efforts of experienced > compiler developers have been spent and are being spent for GCC > development than for LLVM. > > The above said about compilation speed is true when GCC front-end is > used for LLVM. LLVM has another C-language family front-end called > CLANG which can speed up compilation in optimization mode > (-O2/-O3) upto 20%-25%. So even as LLVM optimizations > are not faster than GCC optimizations, CLANG front-end is really > faster than GCC-frontend. I think GCC community should pay more attention > to this fact. Fortunately, a few new GCC projects address to this problem > and I hope this problem will be solved or alleviated. > > This year I used -Ofast -flto -fwhole-program instead of > -O3 for GCC and -O3 -ffast-math for LLVM for comparison of peak > performance. I could improve GCC performance even more by using > other GCC possibilities (like support of AVX insns, Graphite optimizations > and even some experimental stuff like LIPO) but I wanted to give LLVM > some chances too. Probably an experienced user in LLVM could improve > LLVM performance too. So I think it is a fair comparison. >
