On 2022-02-22 16:59, Patrick McGehearty wrote:
I studied Dhrystone about 30 years ago and found it
had a number of flaws back then. For example, most of
the loops in the code are only executed 1-3 times, which
minimizes the value of hoisting values out of inner loops.
Read the Dhrystone wikipedia article for more information.
Going back to what benchmarks might be useful...
you might consider the Livermore Loops
http://www.netlib.org/benchmark/livermorec
These are 24 kernels (tight loops) originally in Fortran but
ported to C 30 years ago. They are reasonably representative
of floating point computational kernels. They are available
without a fee.
Even if you have no interest in floating point computation for
your target architecture, examining the assembly output of these
kernels will be helpful in finding where your port of gcc
is doing well and where the machine architecture input to the
various optimizer phases need some tuning.
You also might review that code and write some modest
test loops of your own for integer code patterns.
Developing good benchmarks is a skill which requires the
developer to know the intended purpose of the benchmark.
I.e. is our goal to compare optimizer implementations?
or different architectures (i.e. arm vs x86)?
or different implementations of an architecture
(i.e. intel vs amd or early x86 vs current x86)
or ...
well, you get the idea.
Good luck,
- Patrick McGehearty
On 2/22/2022 3:49 PM, Paul Koning via Gcc wrote:
On Feb 22, 2022, at 4:26 PM, Gary Oblock via Gcc <gcc@gcc.gnu.org>
wrote:
Andras,
The whole point of benchmarks is to judge a processor's performance.
That being said, just crippling GCC is not reasonable because
processors must be judged in the appropriate context and that
includes the current state of the art compiler technology. If you
have
a new processor I'd benchmark it using the applications you built it
for.
Exactly. Part of what you want to see is that GCC optimizes well for
the new machine, i.e., that there aren't artifacts of the machine
description that get in the way of optimization.
So you'd want to use applications that are good exercises not just of
the code generator but also the optimizer. Dhrystone isn't really
that, because it has evolved into mostly an optimizer test, not a
machine or code generator test.
paul
Thank you for all the feedback. When I said 'crippling' GCC, I didn't
mean to make it generally worse, just to make sure I don't trigger
dhrystone-specific optimizations, if there are any.
Your point about comparing processors is a fair one and that's where I'm
heading, but I need to start small. Even just comparing object file
(.text section) size between architectures have been hugely
enlightening.
There is another point though: my port of GCC is certainly not state of
the art (well, it technically is, because that's the only port in
existence, but you get what I mean). Part of the process is to hone in
the port in terms of tuning the instruction selection, peepholes, etc.
In some sense this work benchmarks compilers not processors. At any
rate, I do get that dhrystone is a poor substitute for a real benchmark
and I'll try to move away from it.
I wasn't aware of Livermore loops being ported to C (and I don't yet
have a Fortran port), so that's a great suggestion. I'll check it out.
Thanks again for all the help,
Andras
