x86_64 unwinder in libgcc_s
Hello! I got strange problem in LuaJIT [1] stack unwinding on Solaris x64. If I build everything using GCC with unwinder from libgcc_s everything works fine. But when I try to use GCC-built libluajit.a in executable built using Sun Studio 12.2 with standard solaris unwinder from libc I got a crash. As it turned out this is due to invalid value that _Unwind_GetCFA() function returned (called from Lua personality routine) E.g. in core I can see the following stack: [1] err_unwind(0x71255e31, 0x6e0bd80, 0x0, 0x6e0bd80, 0x6e0b5d0, 0x7124bbb5), at 0x7124b875 [2] lj_err_unwind_dwarf(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7124bc25 [3] _Unwind_RaiseException_Body(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfd7ffaae0bfc [4] _SUNW_Unwind_RaiseException(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfd7ffaae0de9 [5] err_raise_ext(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7124be2c [6] lj_err_throw(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7124be89 [7] lj_trace_err_info(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7129b8d8 [8] lj_record_ins(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7127f0ec [9] trace_state(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7129d7cd [10] lj_vm_cpcall(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x712473ba [11] lj_trace_ins(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7129db8d [12] lj_dispatch_ins(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x71255e31 [13] lj_vm_inshook(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x71248a41 [14] lua_pcall(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x7125a584 ... lj_err_unwind_dwarf() is personality routine for lj_vm_cpcall() (frame 10) so it's called to process its stack frame. x86_64 psABI says that _Unwind_GetCFA() "returns the 64-bit Canonical Frame Address which is defined as the value of %rsp at the call site in the previous frame." (6.2.5 Context Management) So if I understand this correctly *(CFA - 8) should contain address somewhere inside lj_vm_cpcall() caller, i.e. in lj_trace_ins() (frame 11). And when using Solaris std unwinder it's in fact true, but it crashes. When using GCC unwinder return address is somewhere inside lj_vm_cpcall() itself, i.e. AFAIU CFA for trace_state() (frame 9) is returned instead, but it works. I can make std unwinder work by adjusting returned CFA to point to next frame by subtracting 80 bytes from it but it's interesting to understand why this happens. Am I missing something and GCC unwinder is right? TIA [1] http://luajit.org/
Re: x86_64 unwinder in libgcc_s
On 02.08.2012 20:05, Dmitri Shubin wrote:
Hello!
I got strange problem in LuaJIT [1] stack unwinding on Solaris x64.
I wrote minimal test that reproduces the problem:
$ cat main.c
#include
#include
typedef struct _Unwind_Exception
{
uint64_t exclass;
void (*excleanup)(int, struct _Unwind_Exception);
uintptr_t p1, p2;
} __attribute__((__aligned__)) _Unwind_Exception;
typedef struct _Unwind_Context _Unwind_Context;
extern uintptr_t _Unwind_GetCFA(_Unwind_Context *);
extern int _Unwind_RaiseException(_Unwind_Exception *);
int
foo_personality(int version, int actions, uint64_t uexclass,
_Unwind_Exception *uex, _Unwind_Context *ctx)
{
char *cfa = (char *) _Unwind_GetCFA(ctx);
printf("cfa = %p\nra = %p\n", cfa, *(void **)(cfa - 8));
}
void
throw()
{
static _Unwind_Exception static_uex;
static_uex.exclass = 0x0102030405060708ULL;
static_uex.excleanup = NULL;
_Unwind_RaiseException(&static_uex);
}
extern void foo(void);
int
main()
{
printf("&main = %p, &foo = %p\n", &main, &foo);
foo();
return 0;
}
$ cat foo.s
.file "foo.s"
.text
.globl foo
.type foo, @function
foo:
callthrow
ret
.size foo, .-foo
.section .eh_frame,"a",@unwind
.Lframe1:
.long .LECIE1-.LSCIE1
.LSCIE1:
.long 0
.byte 0x1
.string "zPR"
.uleb128 0x1
.sleb128 -8
.byte 0x10
.uleb128 6
.byte 0x1b
.long foo_personality-.
.byte 0x1b
.byte 0xc
.uleb128 0x7
.uleb128 8
.byte 0x80+0x10
.uleb128 0x1
.align 8
.LECIE1:
.LSFDE2:
.long .LEFDE2-.LASFDE2
.LASFDE2:
.long .LASFDE2-.Lframe1
.long foo-.
.long 6
.uleb128 0
.align 8
.LEFDE2:
First I use GCC (4.7.1 from OpenCSW) to compile both files:
$ /opt/csw/bin/gcc --version | head -1
gcc (GCC) 4.7.1
$ /opt/csw/bin/gcc -m64 main.c foo.s
$ ./a.out
&main = 401834, &foo = 401860
cfa = fd7fffdffaf8
ra = 401865
$ gobjdump -dr a.out
...
00401860 :
401860: e8 a0 ff ff ff callq 401805
401865: c3 retq
...
AFAIU here GCC supplied stack unwinder from statically linked libgcc_s
is used and return address for foo context points to somewhere inside foo()
Second I use GCC (gas in fact) to compile foo.s and SunStudio C to
compile main.c and link:
$ cc -V 2>&1 | head -1
cc: Sun C 5.11 SunOS_i386 145355-03 2011/02/11
$ /opt/csw/bin/gcc -m64 -c foo.s
$ cc -m64 main.c foo.o
$ ./a.out
&main = 400b30, &foo = 400b70
cfa = fd7fffdffaf0
ra = 400b61
$ gobjdump -dr a.out
...
00400b30 :
400b30: 55 push %rbp
400b31: 48 8b ecmov%rsp,%rbp
400b34: 48 83 ec 10 sub$0x10,%rsp
400b38: 48 c7 c2 70 0b 40 00mov$0x400b70,%rdx
400b3f: 48 c7 c6 30 0b 40 00mov$0x400b30,%rsi
400b46: 48 c7 c7 e0 0b 40 00mov$0x400be0,%rdi
400b4d: b8 00 00 00 00 mov$0x0,%eax
400b52: e8 49 fd ff ff callq 4008a0
400b57: b8 00 00 00 00 mov$0x0,%eax
400b5c: e8 0f 00 00 00 callq 400b70
400b61: c7 45 fc 00 00 00 00movl $0x0,-0x4(%rbp)
400b68: 8b 45 fcmov-0x4(%rbp),%eax
400b6b: c9 leaveq
400b6c: c3 retq
400b6d: 00 00 add%al,(%rax)
...
In this case standard Solaris unwinder from libc is used and return
address points to main()
From my understanding of CFA definition given in x86-64 psABI ("the
value of %rsp at the call site in the previous frame." 6.2.5 Context
Management) it looks like value returned by GCC unwinder is wrong.
Am I missing something here?
BTW on Linux (w/ GCC) RA points to foo() as in case 1 so it's
GCC-specific rather than Solaris-specific behavior.
Thanks!
Re: x86_64 unwinder in libgcc_s
On 06.08.2012 21:13, Richard Henderson wrote:
On 08/06/2012 08:23 AM, Dmitri Shubin wrote:
char *cfa = (char *) _Unwind_GetCFA(ctx);
printf("cfa = %p\nra = %p\n", cfa, *(void **)(cfa - 8));
Use _Unwind_GetIP here, for one.
In fact I'm not interested in IP or RA here, I need some context stored
by assembly routine on the stack relative to its CFA.
I use RA only to illustrate that I got "wrong" (from my pov) CFA from
libgcc unwinder.
And the question is is it a bug (or deviation from x86-64 psABI) in
libgcc or I misunderstand what CFA is.
Re: x86_64 unwinder in libgcc_s
Any thoughts on this?
Or maybe it's wrong list for this question?
On 07.08.2012 12:09, Dmitri Shubin wrote:
On 06.08.2012 21:13, Richard Henderson wrote:
On 08/06/2012 08:23 AM, Dmitri Shubin wrote:
char *cfa = (char *) _Unwind_GetCFA(ctx);
printf("cfa = %p\nra = %p\n", cfa, *(void **)(cfa - 8));
Use _Unwind_GetIP here, for one.
In fact I'm not interested in IP or RA here, I need some context
stored by assembly routine on the stack relative to its CFA.
I use RA only to illustrate that I got "wrong" (from my pov) CFA from
libgcc unwinder.
And the question is is it a bug (or deviation from x86-64 psABI) in
libgcc or I misunderstand what CFA is.
Re: x86_64 unwinder in libgcc_s
On 14.08.2012 14:18, Andrew Haley wrote:
You've already had an answer from Richard Henderson, who is probably
the best-placed person to answer you.
My question was: why I get wrong (from my pov) CFA value from GCC unwinder.
I rewritten my small test.
$ cat main.c
#include
#include
typedef struct _Unwind_Exception
{
uint64_t exclass;
void (*excleanup)(int, struct _Unwind_Exception);
uintptr_t p1, p2;
} __attribute__((__aligned__)) _Unwind_Exception;
typedef struct _Unwind_Context _Unwind_Context;
extern uintptr_t _Unwind_GetCFA(_Unwind_Context *);
extern uintptr_t _Unwind_GetIP(_Unwind_Context *);
extern int _Unwind_RaiseException(_Unwind_Exception *);
int
foo_personality(int version, int actions, uint64_t uexclass,
_Unwind_Exception *uex, _Unwind_Context *ctx)
{
char *cfa = (char *) _Unwind_GetCFA(ctx);
char *ip = (char *) _Unwind_GetIP(ctx);
uint64_t v = *(uint64_t *)(cfa - 16);
printf("cfa = %p\nra = %p\nip = %p\nv = %llx\n*cfa = %llx\n", cfa,
*(void **)(cfa - 8), ip, v, *(uint64_t *)cfa);
return 8; // _URC_CONTINUE_UNWIND
}
void
throw()
{
static _Unwind_Exception static_uex;
static_uex.exclass = 0x0102030405060708ULL;
static_uex.excleanup = NULL;
_Unwind_RaiseException(&static_uex);
}
extern void foo(void);
int
main()
{
printf("&main = %p, &foo = %p\n", &main, &foo);
foo();
return 0;
}
$ cat foo.s
.file "foo.s"
.text
.globl foo
.type foo, @function
foo:
mov $0x1020304050,%rax
push%rax
callthrow
pop %rax
ret
.size foo, .-foo
.section .eh_frame,"a",@unwind
.Lframe1:
.long .LECIE1-.LSCIE1
.LSCIE1:
.long 0
.byte 0x1
.string "zPR"
.uleb128 0x1
.sleb128 -8
.byte 0x10
.uleb128 6
.byte 0x1b
.long foo_personality-.
.byte 0x1b
.byte 0xc
.uleb128 0x7
.uleb128 8
.byte 0x80+0x10
.uleb128 0x1
.align 8
.LECIE1:
.LSFDE2:
.long .LEFDE2-.LASFDE2
.LASFDE2:
.long .LASFDE2-.Lframe1
.long foo-.
.long 17
.uleb128 0
.byte 0xe
.uleb128 16
.align 8
.LEFDE2:
As one can see here in foo() I placed constant 0x1020304050 right after
return address to main(), e.g. *(CFA - 16) and in personality routine I
try to get it from there.
First I use standard Solaris unwinder from libc:
$ /opt/csw/bin/gas -64 foo.s
$ cc -m64 main.c foo.o
$ ./a.out
&main = 400bc0, &foo = 400c00
cfa = fd7fffdffb10
ra = 400bf1
ip = 400c10
v = 1020304050
*cfa = 1
And got expected value (v = 1020304050).
Then I use unwinder from libgcc_s:
$ /opt/csw/bin/gcc -m64 main.c foo.s
$ ./a.out
&main = 40186a, &foo = 401894
cfa = fd7fffdffb10
ra = 4018a4
ip = 4018a4
v = fd7fffdffb20
*cfa = 1020304050
And got something totally wrong.
In foo_personality() I expect to have the following stack layout for
foo() frame (width is 8 bytes):
:: ^
| main() frame | |
++ <--- CFA for foo() |
increasing addresses
| return address | <--- points to somewhere in main() |
++ |
| 0x1020304050 | <--- some context value stored by foo() |
++
::
And this is what I have when using Solaris unwinder. But libgcc returned
me another CFA:
:: ^
| main() frame | |
++ | increasing addresses
| return address | <--- points to somewhere in main() |
++ |
| 0x1020304050 | <--- some context value stored by foo() |
++ <--- CFA for foo() returned by libgcc_s
::
So the question is who is right here?
From my understanding of definition of CFA given in x86_64 psABI
Solaris unwinder is right and libgcc_s one is wrong.
See also
http://stackoverflow.com/questions/7534420/gas-explanation-of-cfi-def-cfa-offset
Re: x86_64 unwinder in libgcc_s
On 14.08.2012 17:58, Ian Lance Taylor wrote:
unwinder is right and libgcc_s one is wrong.
I think the definition of _Unwind_GetCFA is ambiguous. It says "the
value of %rsp at the call site in the previous frame." GCC is
returning the value of %rsp at the point of the call to throw.
Solaris is returning the value of %rsp at the call to foo. I don't
know which is correct. _Unwind_GetCFA is not part of the C++
exception handling ABI; I'm not sure where it came from.
_Unwind_GetCFA() is defined in x86_64 psABI:
www.*x86*-*64*.org/documentation/*abi*.pdf
"6.2 Unwind Library Interface"
In my example foo_personality() is set as personality routine for foo()
so I assume the former is called with _Unwind_Context created for foo()
I.e. foo()'s stack frame is 'current' and from this pov the value of
'previous frame' is unambiguous -- it's main()'s frame.
BTW dwarf unwinding instructions generated by GCC use CFA-relative
addresses to locate registers saved on stack frame:
$ cat a.c
void foo()
{
}
$ /opt/csw/bin/gcc -m64 -c a.c
$ gobjdump -dr a.o
a.o: file format elf64-x86-64-sol2
Disassembly of section .text:
:
0: 55 push %rbp
1: 48 89 e5mov%rsp,%rbp
4: 5d pop%rbp
5: c3 retq
$ gobjdump -Wf a.o
a.o: file format elf64-x86-64-sol2
Contents of the .eh_frame section:
0014 CIE
Version: 1
Augmentation: "zR"
Code alignment factor: 1
Data alignment factor: -8
Return address column: 16
Augmentation data: 1b
DW_CFA_def_cfa: r7 (rsp) ofs 8
DW_CFA_offset: r16 (rip) at cfa-8
DW_CFA_nop
DW_CFA_nop
0018 001c 001c FDE cie= pc=..0006
DW_CFA_advance_loc: 1 to 0001
DW_CFA_def_cfa_offset: 16
DW_CFA_offset: r6 (rbp) at cfa-16
DW_CFA_advance_loc: 3 to 0004
DW_CFA_def_cfa_register: r6 (rbp)
DW_CFA_advance_loc: 1 to 0005
DW_CFA_def_cfa: r7 (rsp) ofs 8
DW_CFA_nop
DW_CFA_nop
DW_CFA_nop
Here CIE specifies that right before executing first instruction of
foo() CFA is RSP+8 and return address is stored at offset -8 from CFA.
Why do you want the CFA? Perhaps you can use _Unwind_GetGR or other
functions that are part of the C++ exception interface?
It's not me, it's author of LuaJIT who use CFA to get some state
information from stack frame.
Here is his reply to my question about unwinding problem on Solaris x64
http://www.freelists.org/post/luajit/Stack-unwinding-problem-on-Solaris-x64,1
