Hi Everyone, We are tracking an issue on ppc64-le and GCC 8. We can duplicate it on GCC112 when using /opt/cfarm/gcc8-r257824 compiler. Also see https://github.com/weidai11/cryptopp/issues/588.
We have the issue isolated, but its not really reduced yet. When '-O2 -fstack-protector-strong' is used during a compile then we have some failed self tests. When '-O3 -fstack-protector-strong' is used during a compile then everything is OK. (It is not backwards; -O3 is OK, and -O2 is bad). The source code is clean under the sanitizers on the platform. The problem does not occur under GCC 4.8.5 (default compiler) and GCC 7 (installed at /opt/cfarm/gcc-latest). The source file that is causing the problems under Power8 is https://github.com/weidai11/cryptopp/blob/master/rijndael-simd.cpp#L536 . Its an awful file and nearly everything is inlined. I've tried looking at the disassemblies but I don't have the knowledge to spot the trouble. I'm not sure how to troubleshoot things further. Suggestions on the next step would be very helpful. Thanks in advance.
