A co-worker testing MySQL 4 on NetBSD/amd64 was having trouble with random crashes. He tracked the crashes down to mf_radix.c:radixsort_for_str_ptr(), and handed the problem off to me for further analysis.
I cut down mf_radix.c (attached to this bug report), and looked at the code generated with "gcc -O2 -S mf_radix.c". It appears that the memory for local variables is not accounted for entirely correctly. There are 1024+ bytes of local variables, but the stack pointer (%rsp) is only adjusted by 912 bytes. And it appears local variable "count" is referenced before the stack pointer (-120(%rsp)). I believe, but have not yet confirmed, that the crashes occur because something in NetBSD's thread/task switch code is overwriting the values below the stack pointer assuming that it is free for temporary use. This may also effect other systems --I've done the same in embedded kernels I've written in the past. NetBSD 2.0RC4 uses a modified version of gcc 3.3.3, so I have confirmed that the problem still exists with unmodified gcc 3.3.5. Linking in versions of mf_radix.c compiled without optimization or with gcc 3.4 eliminates the crashes. My attempts to build MySQL 4 entirely with gcc 3.4.X have not been successful, but I have not had enough time to investigate why. -- Summary: generated code uses memory beyond allocated stack frame. Product: gcc Version: 3.3.5 Status: UNCONFIRMED Severity: normal Priority: P2 Component: c AssignedTo: unassigned at gcc dot gnu dot org ReportedBy: jtc at acorntoolworks dot com CC: gcc-bugs at gcc dot gnu dot org GCC host triplet: i386-netbsd GCC target triplet: amd64-netbsd http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18260