OpenSSL Segmentation Faulting (Core dumps included)

Ladar Levison Tue, 05 Apr 2005 08:52:49 -0700

Howdy gents. I wrote an SMTP server using OpenSSL, and it periodically likes to segmentation fault. The core dumps are below. I can't seem to get full stack traces, but I think that is related to the number threads I am using. When I have 128 threads, I get no core dumps. When I have 16 threads, I get the core dumps shown below. Any advice on how to get better dumps would be appreciated. (I've already tried setrlimit.)

The problem only seems be happening once a process has been running for about 24 hours. The majority of the cores indicate a problem with SSL_SESSION_list_remove. Googling doesn't yield any answers.

The version is 0.9.7.f, compiled using ./configure threads -D_REENTRANT; make install.

The machine itself is RHEL3, running kernel 2.4.21-27.0.2.ELsmp. I'd be happy to hear any theories people have. The only two I have are, a) that its some sort of obscure stack smashing bug. Or b) there is a conflict with the RHEL version of OpenSSL loaded on the machine. The ldd output seems to indicate that /lib/libssl.so.4, etc, is linked in, when I know for a fact I've compiled against the libssl.a in the /usr/local/ssl/lib directory. This particular build has that linked in statically. (I think the mysqlclient is linking in the other SSL lib, though it shouldn't make a difference because I don't use it.

Thanks in advance,

L~

[EMAIL PROTECTED] root]# gdb ~/bin/dispatchd cores/core.15774 GNU gdb Red Hat Linux (6.1post-1.20040607.52rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `dispatchd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libmysqlclient_r.so.14...done.
Loaded symbols for /usr/lib/libmysqlclient_r.so.14
Reading symbols from /usr/lib/libbz2.so.1...done.
Loaded symbols for /usr/lib/libbz2.so.1
Reading symbols from /usr/lib/libgmp.so.3...done.
Loaded symbols for /usr/lib/libgmp.so.3
Reading symbols from /usr/lib/libcurl.so.2...done.
Loaded symbols for /usr/lib/libcurl.so.2
Reading symbols from /usr/kerberos/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/kerberos/lib/libgssapi_krb5.so.2
Reading symbols from /usr/kerberos/lib/libkrb5.so.3...done.
Loaded symbols for /usr/kerberos/lib/libkrb5.so.3
Reading symbols from /usr/kerberos/lib/libcom_err.so.3...done.
Loaded symbols for /usr/kerberos/lib/libcom_err.so.3
Reading symbols from /usr/kerberos/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/kerberos/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/tls/libpthread.so.0...done.
Loaded symbols for /lib/tls/libpthread.so.0
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/libmysqlclient.so.14...done.
Loaded symbols for /usr/lib/libmysqlclient.so.14
Reading symbols from /lib/tls/libm.so.6...done.
Loaded symbols for /lib/tls/libm.so.6
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /lib/libssl.so.4...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/libcrypto.so.4...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
#0  0x080ce33c in SSL_SESSION_list_remove ()
(gdb)

[EMAIL PROTECTED] root]# gdb ~/bin/dispatchd core.15781 GNU gdb Red Hat Linux (6.1post-1.20040607.52rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db lib rary "/lib/tls/libthread_db.so.1".

Core was generated by `dispatchd'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/libmysqlclient_r.so.14...done. Loaded symbols for /usr/lib/libmysqlclient_r.so.14 Reading symbols from /usr/lib/libbz2.so.1...done. Loaded symbols for /usr/lib/libbz2.so.1 Reading symbols from /usr/lib/libgmp.so.3...done. Loaded symbols for /usr/lib/libgmp.so.3 Reading symbols from /usr/lib/libcurl.so.2...done. Loaded symbols for /usr/lib/libcurl.so.2 Reading symbols from /usr/kerberos/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/kerberos/lib/libgssapi_krb5.so.2 Reading symbols from /usr/kerberos/lib/libkrb5.so.3...done. Loaded symbols for /usr/kerberos/lib/libkrb5.so.3 Reading symbols from /usr/kerberos/lib/libcom_err.so.3...done. Loaded symbols for /usr/kerberos/lib/libcom_err.so.3 Reading symbols from /usr/kerberos/lib/libk5crypto.so.3...done. Loaded symbols for /usr/kerberos/lib/libk5crypto.so.3 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/tls/libpthread.so.0...done. Loaded symbols for /lib/tls/libpthread.so.0 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /usr/lib/libmysqlclient.so.14...done. Loaded symbols for /usr/lib/libmysqlclient.so.14 Reading symbols from /lib/tls/libm.so.6...done. Loaded symbols for /lib/tls/libm.so.6 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /lib/libnss_dns.so.2...done. Loaded symbols for /lib/libnss_dns.so.2 Reading symbols from /lib/libssl.so.4...done. Loaded symbols for /lib/libssl.so.4 Reading symbols from /lib/libcrypto.so.4...done. Loaded symbols for /lib/libcrypto.so.4 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 0x080ce702 in timeout_LHASH_DOALL_ARG () (gdb) quit [EMAIL PROTECTED] root]# gdb ~/bin/dispatchd core.15791 GNU gdb Red Hat Linux (6.1post-1.20040607.52rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `dispatchd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libmysqlclient_r.so.14...done.
Loaded symbols for /usr/lib/libmysqlclient_r.so.14
Reading symbols from /usr/lib/libbz2.so.1...done.
Loaded symbols for /usr/lib/libbz2.so.1
Reading symbols from /usr/lib/libgmp.so.3...done.
Loaded symbols for /usr/lib/libgmp.so.3
Reading symbols from /usr/lib/libcurl.so.2...done.
Loaded symbols for /usr/lib/libcurl.so.2
Reading symbols from /usr/kerberos/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/kerberos/lib/libgssapi_krb5.so.2
Reading symbols from /usr/kerberos/lib/libkrb5.so.3...done.
Loaded symbols for /usr/kerberos/lib/libkrb5.so.3
Reading symbols from /usr/kerberos/lib/libcom_err.so.3...done.
Loaded symbols for /usr/kerberos/lib/libcom_err.so.3
Reading symbols from /usr/kerberos/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/kerberos/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/tls/libpthread.so.0...done.
Loaded symbols for /lib/tls/libpthread.so.0
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/libmysqlclient.so.14...done.
Loaded symbols for /usr/lib/libmysqlclient.so.14
Reading symbols from /lib/tls/libm.so.6...done.
Loaded symbols for /lib/tls/libm.so.6
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /lib/libssl.so.4...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/libcrypto.so.4...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
#0  0x080d64ee in CRYPTO_add_lock ()
(gdb) quit

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    openssl-users@openssl.org
Automated List Manager                           [EMAIL PROTECTED]

OpenSSL Segmentation Faulting (Core dumps included)

Reply via email to