On Thu, Jan 23, 2014 at 11:52:00PM -0500, Vladimir Makarov wrote: > o IMHO, the data in articles lack credability may be because a wrong > setup (by me or by phoronix). E.g. I tried to reproduce Scimark > results for GCC4.8 and LLVM3.3 from his article "LLVM Clang 3.4 > Already Has Some Performance Changes": > > http://www.phoronix.com/scan.php?page=article&item=llvm_clang34_first&num=2 > > Phoronix used i7-4770K for this. I used the closest machine I found > i5-4670 (with switched turbo mode off). The important difference is > 0.1Ghz in frequency (3.5Ghz vs. 3.4 Ghz). I got GCC Scimark > (-large) composite score close to the article when I used -O and > still on my machine the composite score was 20% higher than the > article reports although the article says that -O3 -march=core-avx > were used.
Yeah, that is my experience too, e.g. on http://www.phoronix.com/scan.php?page=article&item=llvm34_gcc49_compilers&num=3 Phoronix claims the LU benchmark improved 80% with LLVM 3.4 and ahead of GCC, but I couldn't reproduce anything close to that, while LLVM 3.4 slightly improved compared to LLVM 3.3, it was still comparable to GCC 4.8 and behind GCC 4.9. All that matters in the benchmark is a single loop though: for (jj=j+1; jj<N; jj++) Aii[jj] -= AiiJ * Aj[jj]; vectorized by all tested compilers, so clearly a microbenchmark. It is not that long that Phoronix used to compile some of the benchmarks e.g. with -O0 and claim they were compiled with -O3, or tune for a completely different CPU than what it has been tested on, their articles often comment on the numbers in quite biased way without actually bothering to look at why are the numbers changing, what number changes are within the noise and if something changes more than that and is repeatable, why it happened, so their numbers and benchmarks can't be considered the most credible benchmarks out there. Jakub
