I wouldn't advocate a neural network within FPC since, yes, it would be
unbelievably slow, very difficult to port and even more difficult to
maintain and verify... not just to confirm correct output (e.g. BSR is
good as an approximation to the expression "floor(log2(x))", but it has
some awkwardness if the input is zero) but also to ensure something
undesired (i.e. a backdoor) hasn't been slipped inside the assembly
language. I would be curious to see a neural net be able to improve on
small algorithms though.
My slight concern is that the x86 peephole optimizer is becoming pretty
bloated now, mostly thanks to my slightly haphazard development (I see a
source of improvement in a particular set of circumstances), and while
studying a diff of the output, I see another block of assembly language
nearby and think 'that could be improved too', and the end result is a
massive jungle of "if... then" blocks, which I suppose is the ultimate
building block of artificial intelligence (one for Sven... "The Diamond
Age" by Neal Stephenson has characters use the term "pseudo
intelligence" instead, with the characters remarking that it's more
accurate!)
Ultimately I want to look for cleaner and more fundamental solutions
since each new feature makes the compiler more bloated. Then again, if
you compare the speed and size of the generated binaries between the
trunk and a couple of releases ago, the difference is paramount. I
can't remember the proverb that Florian used (in a sense, approving of
small peephole optimizer changes), but one I learned in Chinese martial
arts would be "the journey of a thousand miles starts with a single
step"... lots and lots of small changes that individually are almost
pointless, but which build up over time and when you look back at the
starting point, the difference becomes very apparent.
To get back to the point, I wouldn't advocate giving FPC a neural
network, or any compiler really except as a development aid, with
generated results having an implicit "hey, might this work?" for the
developer. Random instruction changes doesn't feel particularly robust
though, and I would wonder if a neural net would work out how to change,
say "y := x div 3;" to "y := High32Bits(x * $AAAAAAAB) shr 1" ...
essentially turning a division by a constant into a multiplication by
its reciprocal, which a lot of compilers use (including FPC on x86 and
AArch64) to get around the integer division instruction being rather slow.
Kit
On 10/11/2022 23:08, Karoly Balogh via fpc-devel wrote:
Hi,
On Thu, 10 Nov 2022, Sven Barth via fpc-devel wrote:
You still need to feed the model with the necessary rules and with
necessary training data of both correct and incorrect approaches.
But even then *I* wouldn't want to have any of that black box mambo
jumbo in FPC, cause when a bug occurs in the optimizations due to some
decision the model made... well... tough luck.
Well, a few years back I read a paper (maybe it was linked before in the
thread no idea), that embraced this, and used a genetic algorithm for
optimization, basically, took the default compiler output, and just
started to change instructions randomly to see if it performed better...
99,99999% cases it crashed, but eventually there was a version that
performed better. So for the next iteration that was taken as baseline.
(Of course, with the necessary state-verifications surrounding the code.)
Now, I could still see a billion ways this could fall apart (what if you
change input data, so you have to combine it with some fuzzing too for
sure?), but it was an interesting idea nevertheless. And of course,
unbelievably slow.
Anyway, until a simple memory layout change and pure chances of caching in
a live system change can cause huge differences in code performance,
sometimes bigger changes than the optimizer can make(!) the entire
excercise is largely pointless, IMO. As much as one of course don't want
to generate total junk as code, in real world applications there are
usually bigger wins to be had just by improving the used algorithms.
(I've seen a talk about this once, where they basically measured GCC's
optimizer against caching and code layout changes, and the optimizer's
effect was mostly within measurement error, apart from a few big things
like register variables... Which is nuts.)
Charlie
_______________________________________________
fpc-devel maillist - fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
_______________________________________________
fpc-devel maillist - fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
