Hi Ryan,
First of all, thanks for your investigation.
I've reverted for now the patch in SOPE (from PR#4). We might want to
re-integrate this change once libldap is fixed.
Thanks,
On 29/10/2015 23:53, Ryan Tandy wrote:
On Tue, Oct 27, 2015 at 02:54:10PM -0700, Ryan Tandy wrote:
Can you please generate that trace again with libgnutls28-dbg
installed, so that we can see more details?
#0 close () at ../sysdeps/unix/syscall-template.S:81
#1 0x00007f2d3a300013 in _rnd_system_entropy_deinit () at
rnd-common.c:254
#2 0x00007f2d3a300126 in wrap_nettle_rnd_deinit (ctx=<optimized out>)
at rnd.c:157
#3 0x00007f2d3a2612b6 in _gnutls_global_deinit (destructor=0) at
gnutls_global.c:364
#4 0x00007f2d3a27225f in gnutls_global_set_mutex (init=0x3,
deinit=0x0, lock=0x7f2d3a53ce08 <gnutls_rnd_ctx>,
unlock=0x7f2d391bbaa0 <tlsg_mutex_unlock>) at locks.c:60
#5 0x00007f2d391b9d66 in tls_init (impl=0x7f2d393d2440
<ldap_int_tls_impl>) at tls2.c:170
#6 0x00007f2d391bae53 in ldap_int_tls_start
(ld=ld@entry=0x7f2d3fe88b60, conn=conn@entry=0x7f2d3fe91fd0,
srv=srv@entry=0x7f2d3fe889b0) at tls2.c:838
#7 0x00007f2d39194f6d in ldap_int_open_connection (ld=0x7f2d3fe88b60,
conn=0x7f2d3fe91fd0, srv=0x7f2d3fe889b0,
async=<optimized out>) at open.c:448
#8 0x00007f2d391a81bd in ldap_new_connection (ld=0x7f2d3fe88b60,
srvlist=0x0, srvlist@entry=0x7f2d3fe88ca8,
use_ldsb=978570760, use_ldsb@entry=1, connect=0, bind=0x0, m_req=0,
m_res=0) at request.c:484
#9 0x00007f2d3919464a in ldap_open_defconn
(ld=ld@entry=0x7f2d3fe88b60) at open.c:41
#10 0x00007f2d391a95e8 in ldap_send_initial_request
(ld=0x7f2d3fe88b60, msgtype=<optimized out>, dn=<optimized out>,
ber=0x7f2d3fe90f80, msgid=1) at request.c:130
#11 0x00007f2d3919def5 in ldap_sasl_bind (ld=0x7f2d3fe88b60,
dn=0x7f2d3fe90f28 "uid=ryan,dc=example,dc=com",
mechanism=0x0, cred=0x7ffe77d8a7f0, sctrls=0x7f2d3a53bfa0
<global_init_mutex>, cctrls=0x7f2d3fe90f80,
msgidp=0x7ffe77d8a784) at sasl.c:148
#12 0x00007f2d3919e46b in ldap_sasl_bind_s (ld=0x7f2d3fe88b60,
dn=0x7f2d3fe90f28 "uid=ryan,dc=example,dc=com",
mechanism=0x0, cred=0x7ffe77d8a7f0, sctrls=0x0,
cctrls=0x7ffe77d8a2f0, servercredp=0x0) at sasl.c:182
#13 0x00007f2d3919ecd2 in ldap_simple_bind_s (ld=0x7f2d3fe88b60,
dn=0x7f2d3fe90f28 "uid=ryan,dc=example,dc=com",
passwd=0x7f2d3fe90f68 "test") at sbind.c:113
http://sources.debian.net/src/gnutls28/3.3.8-6%2Bdeb8u3/lib/nettle/rnd-common.c/#L254
http://sources.debian.net/src/gnutls28/3.3.8-6%2Bdeb8u3/lib/locks.c/#L60
So that's very much not great, and worth looking at again from
upstream PoV whether we actually need to be doing the
gnutls_global_set_mutex stuff. (Especially in light of the comment
above it: "Do not call this function from a library [...]") This in
some ways a recurrence of the "whose job is it to initialize libraries
with global state?" problem that we had with gcrypt, not so long ago.
The really problematic bit, though, is this:
https://github.com/inverse-inc/sope/blob/e870145b7ed381c1bdef5945075ed48948b86d5b/sope-appserver/NGObjWeb/WOWatchDogApplicationMain.m#L1014
The order of operations here is:
1. libraries are loaded
2. gnutls' constructor does the global init, open("/dev/urandom") -> 3
3. WOWatchDogApplicationMain runs, closes everything
4. libldap goes to set up gnutls
5. gnutls prepares to re-init itself, closes the remembered urandom fd
6. things go downhill from here.
Actually this exact scenario is mentioned on a gnutls author's blog:
http://nmav.gnutls.org/2014/12/a-quick-overview-of-gnutls-development.html
There were some patches pulled into gnutls in jessie to address an
application closing the urandom socket and then calling
gnutls_global_init again, but no such handling around
gnutls_global_*de*init.
http://sources.debian.net/src/gnutls28/3.3.8-6%2Bdeb8u3/debian/patches/35_recheck_urandom_fd.diff/
So: this could be addressed in libldap, by not calling
gnutls_global_set_mutex any more, or in sope, by not closing fds so
aggressively at startup. I admit I didn't look as deeply at the
history of that code in libldap as I could have, when I updated it for
gnutls28; I'll do that now.
Ludovic, thoughts from your end?
--
Ludovic Marcotte
[email protected] :: +1.514.755.3630 :: http://inverse.ca
Inverse inc. :: Leaders behind SOGo (http://sogo.nu) and PacketFence
(http://packetfence.org)
