------- Comment #4 from xxcv07 at gmail dot com 2009-08-29 09:33 ------- Hi,
> I built an x86_64-w64-mingw32 cross compiler under x86_64 linux using > latest gcc SVN code, then use this cross compiler to build ffmpeg. I can confirm this issue and encountered this problem after cross compiled VLC with x86_64-w64-mingw32-gcc-4.4.1 from linux the resulting binary will crash mysteriously whenever a file is played, yesterday I found out when vlc is configured with --disable-optimizations and compile proceed with -O0 then the resulting binary won't crash anymore. I can't get any useful backtrace from GDB 64bits. I have looked through Process explorer and was clueless as to why it crashed. > > the program runs failed, it seem that epilogue code in aac_encode_frame() > adjust wrong rsp register. > > at function aac_encode_frame() begin : > > 0x69fb91 <aac_encode_frame+0>: push %rbp > 0x69fb91 <aac_encode_frame+1>: mov %rsp,%rbp > 0x69fb94 <aac_encode_frame+4>: push %r15 > 0x69fb96 <aac_encode_frame+6>: push %r14 > 0x69fb98 <aac_encode_frame+8>: push %r13 > 0x69fb9a <aac_encode_frame+10>: push %r12 > 0x69fb9c <aac_encode_frame+12>: push %rdi > 0x69fb9d <aac_encode_frame+13>: push %rsi > 0x69fb9e <aac_encode_frame+14>: push %rbx > 0x69fb9f <aac_encode_frame+15>: sub $0x178,%rsp > 0x69fba6 <aac_encode_frame+22>: mov %rdx,0x18(%rbp) > 0x69fbaa <aac_encode_frame+26>: mov %rcx,0x10(%rbp) > > it push 7 registers, when it about to quit : > > 0x69fd65 <aac_encode_frame+469>: lea -0x60(%rbp),%rsp > 0x69fd69 <aac_encode_frame+473>: movdqa 0x150(%rsp),%xmm6 > 0x69fd72 <aac_encode_frame+482>: movdqa 0x160(%rsp),%xmm7 > 0x69fd7b <aac_encode_frame+491>: add $0x20,%rsp > 0x69fd7f <aac_encode_frame+495>: pop %rbx > 0x69fd80 <aac_encode_frame+496>: pop %rsi > 0x69fd81 <aac_encode_frame+497>: pop %rdi > 0x69fd82 <aac_encode_frame+498>: pop %r12 > 0x69fd84 <aac_encode_frame+500>: pop %r13 > 0x69fd86 <aac_encode_frame+502>: pop %r14 > 0x69fd88 <aac_encode_frame+504>: pop %r15 > 0x69fd8a <aac_encode_frame+506>: leaveq > 0x69fd8b <aac_encode_frame+507>: retq > > it should "add 0x28 %rsp ( not 0x20 )" after "lea -0x60(%rbp),%rsp" > > > the compile command is : > > /compile/mingw-w64-dgn/cross/bin/x86_64-w64-mingw32-gcc -DHAVE_AV_CONFIG_H -I. > -I"/compile/mingw-w64-dgn/lib_source/ffmpeg" -D_ISOC99_SOURCE > -D_POSIX_C_SOURCE=200112 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -std=c99 > -fno-common -fomit-frame-pointer -g -Wdeclaration-after-statement -Wall > -Wno-switch -Wdisabled-optimization -Wpointer-arith -Wredundant-decls > -Wno-pointer-sign -Wcast-qual -Wwrite-strings -Wtype-limits -Wundef -O3 > -fno-strict-aliasing -fno-math-errno -fno-signed-zeros -fno-tree-vectorize > > -MMD -MF libavcodec/aacenc.d -MT libavcodec/aacenc.o -c -o a.o -save-temps > /compile/mingw-w64-dgn/lib_source/ffmpeg/libavcodec/aacenc.c > > the -save-temps output is attached. > > -O1 -O2 -O3 all has this problem, -O0 has no problem. Now that I see you have posted this issue we can confirm that gcc is indeed producing the wrong code with internal optimization flags, I tested dll which will start to crash when any -O flag is present. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41184
