AIX OpenSSL 9.7d accept fail, peek_last_error says 0, info_callback ret is -1

[d . mclellan]

I'm digging into an AIX SSL_accept failure with no useful error information:  Here's the client/server side state change sequence. where and ret are the corresponding values of these arguments given to the info_callback function.   These state entries were collected using the info_callback routine.   The -ret- argument of info_callback is documented:  0 indicates a failure (implying non-zero indicates still ok).  The server entries for 'Write key exchange A' at the end show ret=negative1 -- generally considered to indicate failure, but I am not sure of the value in this context. What might this mean? 

 

[Server  : SSL State: before/accept initialization,  where=00000010,ret=00000001
[Server  : SSL State: before/accept initialization,  where=00002001,ret=00000001

[Client  : SSL State: before/connect initialization, where=00000010,ret=00000001
[Client  : SSL State: before/connect initialization, where=00001001,ret=00000001
[Client  : SSL State: SSLv2/v3 write client hello A, where=00001001,ret=00000001

[Server  : SSL State: SSLv3 read client hello A,     where=00002001,ret=00000001
[Server  : SSL State: SSLv3 write server hello A,    where=00002001,ret=00000001
[Server  : SSL State: SSLv3 write certificate A,     where=00002001,ret=00000001

 

This first entry comes immediately after a successful return from my DH callback routine

 

[Server  : SSL State: SSLv3 write key exchange A,    where=00002002,ret=ffffffff
[Server  : SSL State: SSLv3 write key exchange A,    where=00002002,ret=ffffffff

[Server  : SSL_accept returned=-1
[Server  : SSL_accept: ERR_peek_last_error shows 0

Thanks again for any help.

 

--
Dave McLellan
EMC Corporation