Hello, Here are the results of relayd debug:
Version: relayd --current OS: OpenBSD 5.3/amd64 pwd: /usr/src/usr.sbin/relayd $ sudo ./relayd $ ps aux|grep relay _relayd 10140 0.0 0.1 1180 2984 ?? S 11:37AM 0:00.01 relayd: relay (relayd) _relayd 32389 0.0 0.1 1184 2996 ?? S 11:37AM 0:00.01 relayd: relay (relayd) root 17108 0.0 0.1 1336 3252 ?? Ss 11:37AM 0:00.02 relayd: parent (relayd) _relayd 29445 0.0 0.1 1136 2744 ?? S 11:37AM 0:00.01 relayd: pfe (relayd) _relayd 10876 0.0 0.1 940 2496 ?? S 11:37AM 0:00.01 relayd: hce (relayd) andu 5868 0.0 0.0 228 124 p0 R+/0 11:37AM 0:00.00 grep relay PFE: $ sudo gdb ./relayd GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-unknown-openbsd5.3"... (gdb) (gdb) attach 29445 Attaching to program: /usr/src/usr.sbin/relayd-current/relayd, process 29445 Loaded symbols for /usr/src/usr.sbin/relayd-current/relayd Reading symbols from /usr/lib/libevent.so.3.1...done. Loaded symbols for /usr/lib/libevent.so.3.1 Reading symbols from /usr/lib/libssl.so.19.0...done. Loaded symbols for /usr/lib/libssl.so.19.0 Reading symbols from /usr/lib/libcrypto.so.22.0...done. Loaded symbols for /usr/lib/libcrypto.so.22.0 Reading symbols from /usr/lib/libutil.so.11.4...done. Loaded symbols for /usr/lib/libutil.so.11.4 Reading symbols from /usr/lib/libc.so.66.2...done. Loaded symbols for /usr/lib/libc.so.66.2 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so [Switching to thread 1029445] 0x000005462804581a in kevent () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> Current language: auto; currently asm (gdb) continue Continuing. Program exited normally. (gdb) The program is not being run. (gdb) HCE: $ sudo gdb ./relayd GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-unknown-openbsd5.3"... (gdb) attach 10876 Attaching to program: /usr/src/usr.sbin/relayd-current/relayd, process 10876 Loaded symbols for /usr/src/usr.sbin/relayd-current/relayd Reading symbols from /usr/lib/libevent.so.3.1...done. Loaded symbols for /usr/lib/libevent.so.3.1 Reading symbols from /usr/lib/libssl.so.19.0...done. Loaded symbols for /usr/lib/libssl.so.19.0 Reading symbols from /usr/lib/libcrypto.so.22.0...done. Loaded symbols for /usr/lib/libcrypto.so.22.0 Reading symbols from /usr/lib/libutil.so.11.4...done. Loaded symbols for /usr/lib/libutil.so.11.4 Reading symbols from /usr/lib/libc.so.66.2...done. Loaded symbols for /usr/lib/libc.so.66.2 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so [Switching to thread 1010876] 0x000005462804581a in kevent () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> Current language: auto; currently asm (gdb) (gdb) continue Continuing. Program exited normally. (gdb) The program is not being run. (gdb) The program is not being run. PARENT: $ sudo gdb ./relayd GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-unknown-openbsd5.3"... (gdb) (gdb) attach 17108 Attaching to program: /usr/src/usr.sbin/relayd-current/relayd, process 17108 Loaded symbols for /usr/src/usr.sbin/relayd-current/relayd Reading symbols from /usr/lib/libevent.so.3.1...done. Loaded symbols for /usr/lib/libevent.so.3.1 Reading symbols from /usr/lib/libssl.so.19.0...done. Loaded symbols for /usr/lib/libssl.so.19.0 Reading symbols from /usr/lib/libcrypto.so.22.0...done. Loaded symbols for /usr/lib/libcrypto.so.22.0 Reading symbols from /usr/lib/libutil.so.11.4...done. Loaded symbols for /usr/lib/libutil.so.11.4 Reading symbols from /usr/lib/libc.so.66.2...done. Loaded symbols for /usr/lib/libc.so.66.2 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so [Switching to thread 1017108] 0x000005462804581a in kevent () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> Current language: auto; currently asm (gdb) (gdb) continue Continuing. Program received signal SIGTERM, Terminated. 0x000005462805feea in kill () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> (gdb) Continuing. Program exited normally. (gdb) The program is not being run. (gdb) The program is not being run. CHILD #1: $ sudo gdb ./relayd GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-unknown-openbsd5.3"... (gdb) attach 32389 Attaching to program: /usr/src/usr.sbin/relayd-current/relayd, process 32389 Loaded symbols for /usr/src/usr.sbin/relayd-current/relayd Reading symbols from /usr/lib/libevent.so.3.1...done. Loaded symbols for /usr/lib/libevent.so.3.1 Reading symbols from /usr/lib/libssl.so.19.0...done. Loaded symbols for /usr/lib/libssl.so.19.0 Reading symbols from /usr/lib/libcrypto.so.22.0...done. Loaded symbols for /usr/lib/libcrypto.so.22.0 Reading symbols from /usr/lib/libutil.so.11.4...done. Loaded symbols for /usr/lib/libutil.so.11.4 Reading symbols from /usr/lib/libc.so.66.2...done. Loaded symbols for /usr/lib/libc.so.66.2 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so [Switching to thread 1032389] 0x000005462804581a in kevent () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> Current language: auto; currently asm (gdb) continue Continuing. Program received signal SIGSEGV, Segmentation fault. event_queue_insert (base=0x0, ev=0x54621afd208, queue=8) at /usr/src/lib/libevent/event.c:997 997 base->event_count++; Current language: auto; currently c (gdb) Continuing. Program terminated with signal SIGSEGV, Segmentation fault. The program no longer exists. (gdb) The program is not being run. (gdb) The program is not being run. CHILD #2: $ sudo gdb ./relayd GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-unknown-openbsd5.3"... (gdb) attach 10140 Attaching to program: /usr/src/usr.sbin/relayd-current/relayd, process 10140 Loaded symbols for /usr/src/usr.sbin/relayd-current/relayd Reading symbols from /usr/lib/libevent.so.3.1...done. Loaded symbols for /usr/lib/libevent.so.3.1 Reading symbols from /usr/lib/libssl.so.19.0...done. Loaded symbols for /usr/lib/libssl.so.19.0 Reading symbols from /usr/lib/libcrypto.so.22.0...done. Loaded symbols for /usr/lib/libcrypto.so.22.0 Reading symbols from /usr/lib/libutil.so.11.4...done. Loaded symbols for /usr/lib/libutil.so.11.4 Reading symbols from /usr/lib/libc.so.66.2...done. Loaded symbols for /usr/lib/libc.so.66.2 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so [Switching to thread 1010140] 0x000005462804581a in kevent () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> Current language: auto; currently asm (gdb) continue Continuing. Program received signal SIGPIPE, Broken pipe. 0x000005462805eb8a in write () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> (gdb) Continuing. Program received signal SIGTERM, Terminated. 0x000005462804581a in kevent () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> (gdb) Continuing. Program received signal SIGTERM, Terminated. 0x000005462804581a in kevent () at <stdin>:2 2 in <stdin> (gdb) Continuing. Program exited normally. (gdb) The program is not being run. It seems that child with pid 32389 receives a SIGSEGV signal, amking the other threads to crash. But why this child crashes in /usr/src/lib/libevent/event.c, line 997 ? If the message is unreadable I can re-send it in attachement ________________________________ From: Chris Cappuccio <ch...@nmedia.net> To: Bogdan Andu <bo...@yahoo.com> Cc: Stuart Henderson <s...@spacehopper.org>; "misc@openbsd.org" <misc@openbsd.org> Sent: Friday, August 23, 2013 8:45 AM Subject: Re: relayd crash You can't debug relayd without attaching to all of the processes (you must use multiple simultaneous gdb sessions) Bogdan Andu [bo...@yahoo.com] wrote: > ok, > > I checked out relayd -current, compiled with debug symbols, launched gdb > and attached to pfe pid : > > pwd: /usr/src/usr.sbin/relayd > > $ sudo gdb ./relayd > GNU gdb 6.3 > Copyright 2004 Free Software Foundation, Inc. > GDB is free > software, covered by the GNU General Public License, and you are > welcome to > change it and/or distribute copies of it under certain conditions. > Type "show > copying" to see the conditions. > There is absolutely no warranty for GDB.? Type > "show warranty" for details. > This GDB was configured as > "amd64-unknown-openbsd5.3"... > (gdb) attach 27726 > Attaching to program: > /usr/src/usr.sbin/relayd/relayd, process 27726 > Loaded symbols for > /usr/src/usr.sbin/relayd/relayd > Reading symbols from > /usr/lib/libevent.so.3.1...done. > Loaded symbols for /usr/lib/libevent.so.3.1 > Reading symbols from /usr/lib/libssl.so.19.0...done. > Loaded symbols for > /usr/lib/libssl.so.19.0 > Reading symbols from > /usr/lib/libcrypto.so.22.0...done. > Loaded symbols for > /usr/lib/libcrypto.so.22.0 > Reading symbols from > /usr/lib/libutil.so.11.4...done. > Loaded symbols for /usr/lib/libutil.so.11.4 > Reading symbols from /usr/lib/libc.so.66.2...done. > Loaded symbols for > /usr/lib/libc.so.66.2 > Reading symbols from /usr/libexec/ld.so...done. > Loaded > symbols for /usr/libexec/ld.so > [Switching to thread 1027726] > 0x0000029e250da81a in kevent () at <stdin>:2 > 2?????? <stdin>: No such file or > directory. > ??????? in <stdin> > Current language:? auto; currently asm > (gdb) > launched qualys scan and now waiting for crash to occur - will take 2 hours. > Unfortunately the relayd patch branch failed with the same error, and that is > why I tried relayd -current > > > I'll come back with crash results > ________________________________ > From: Stuart Henderson <s...@spacehopper.org> > To: misc@openbsd.org > Sent: Wednesday, August 21, 2013 3:51 PM > Subject: Re: > relayd crash > > > On 2013-08-21, Bogdan Andu <bo...@yahoo.com> wrote: > > I tried > relayd -current and crashed with the error: > > pwd: > > /usr/src/usr.sbin/relayd > > > > $ sudo ./relayd -f /etc/relayd.conf -d > > ..... A LOT > > OF LOGS > ................ > > relay www_ssl, session 1368 (1 active), 0, > > 64.39.111.90 > -> :0, SSL accept error > > relay www_ssl, session 1369 (1 active), > > 0, > 64.39.111.90 -> 127.0.0.1:8080, done > > pfe exiting, pid 8360 > > lost child: > > > relay terminated; signal 11 > > hce exiting, pid 18726 > > parent terminating, pid > > 2704 > > Build it with debug symbols (make clean, make DEBUG=-g install) > and > re-run it, then you can attach gdb to the pfe process (use ps > to find the > correct pid and 'gdb /usr/sbin/relayd' and 'attach <pid>'. > Now when you > trigger the crash you should be able to get a backtrace. -- Semicolons; use them
