Hello Matt,
I call the WSAGetLastError() for Windows and that returns 183
(ERROR_ALREADY_EXISTS) //Cannot create a file when that file already exists
The SSL_get_error() gives us SSL_ERROR_SYSCALL
*Server *code is roughly like below
SSL_CTX *m_pCtx;
SSL *m_pSsl;
m_pCtx = SSL_CTX_new(TLS_server_method();
if ((dwRet = LoadCertificates()) != rSUCCESS)
throw dwRet;
if ((m_pSsl = SSL_new(m_pCtx)) != NULL)
{
if ((iRet = SSL_set_fd(m_pSsl, (*this)())) == 0) /* attach the socket
descriptor */
{
sslError = SSL_get_error(m_pSsl, iRet);
LOGERROR(szLine);
throw eSSL_ERROR;
}
SSL_set_info_callback(m_pSsl, apps_ssl_info_callback);
ERR_clear_error();
if ((sslError = SSL_accept(m_pSsl)) < 1)
{
sslError = SSL_get_error(m_pSsl, sslError);
dwRet = handleError(sslError, "SSL_accept failed with error ",
iRet);
throw dwRet;// eSSL_ERROR;
}
}
Client
SSL_CTX *m_pCtx;
SSL *m_pSsl;
m_pCtx = SSL_CTX_new(TLS_client_method();
if ((dwRet = LoadCertificates(TRUE)) != rSUCCESS) //Trust certificates only
throw dwRet;
/* Set for server verification*/
SSL_CTX_set_verify(m_pCtx, SSL_VERIFY_PEER, NULL); //Work in progress
m_pSsl = SSL_new(m_pCtx);
if ((iRet = SSL_set_fd(m_pSsl, (*this)())) == 0) /* attach the socket
descriptor */
{
ssl_error = SSL_get_error(m_pSsl, iRet);
LOGERROR(szLine);
throw eSSL_ERROR;
}
SSL_set_info_callback(m_pSsl, apps_ssl_info_callback);
ERR_clear_error();
if ((iRet = SSL_connect(m_pSsl)) <= 0) /* perform the connection */
{
ssl_error = SSL_get_error(m_pSsl, iRet);
dwRet = handleError(iRet, "SSL_connect failed with error ", ssl_error);
throw eSSL_ERROR;
}
ShowCerts();
}
As mentioned before this code works fine when called by another
application. So the certificates are all valid. I also tried this on
different machines but it did not work- I get the same error.
Thanks
Kamala
On Fri, Feb 4, 2022 at 12:20 PM Matt Caswell <m...@openssl.org> wrote:
> Does errno give you anything?
>
> How did you create your BIOs for m_pSsl?
>
> Matt
>
> On 04/02/2022 16:25, Kamala Ayyar wrote:
> > Hello Matt,
> >
> > The SSL_get_error() returns 5(SSL_ERROR_SYSCALL) It does not print
> > anything for this error, just an empty string.
> > I use the following to print error but nothing is printed
> > if ((retVal = SSL_accept(m_pSsl)) < 1)
> > {
> > sslError = SSL_get_error(m_pSsl, retVal);
> > LOGERROR(getOpenSSLError());
> > throw dwRet;// eSSL_ERROR;
> > }
> > string getOpenSSLError()
> > {
> > BIO *bio = BIO_new(BIO_s_mem());
> > ERR_print_errors(bio);
> > char *buf;
> > size_t len = BIO_get_mem_data(bio, &buf);
> > string ret(buf, len);
> > BIO_free(bio);
> > return ret;
> > }
> >
> > *Kamala Ayyar*
> > 502 Claremont Ave.
> > Teaneck NJ 07666-2563
> > Tel: (201)530-0861
> >
> >
> > On Fri, Feb 4, 2022 at 10:54 AM Matt Caswell <m...@openssl.org
> > <mailto:m...@openssl.org>> wrote:
> >
> >
> >
> > On 04/02/2022 15:17, Kamala Ayyar wrote:
> > >
> > > Hello,
> > >
> > > We are facing a strange handshake failure issue with a test
> > server and
> > > client application using OpenSSL in Windows. We have tried with
> > both
> > > 1.1.1g and 3.0.1 versions- same problem. We created a Dll to
> > handle the
> > > OpenSSL functions- where the SSL context, SSL object and
> > certificates
> > > are handled. The certificates are obtained from the Windows store
> > and
> > > converted to cert and key using PKCS12_parse()
> > > The server accepts non secure connection from the client and then
> > passes
> > > the socket to the Dll that calls the TLS_server_method() and
> > creates the
> > > SSL context, SSL object and loads the certificates for use. It
> > however
> > > fails at SSL_accept(m_pSsl). We use a call
> > > back SSL_set_info_callback(m_pSsl, apps_ssl_info_callback) that
> > gave us
> > > the following error information
> > > SSL_accept:Error in before SSL initialization
> > > On the client side the same Dll is called with a client
> > > method TLS_client_method() and the error displayed
> > is SSL_connect:Error
> > > in SSLv3/TLS write client hello
> > > We have confirmed the certificates are good and valid.
> > >
> > > The same Dll called from a different heavily threaded application
> > with
> > > over 2000+ clients works well and handshake connections
> established
> > > without issues on a different port number.
> > >
> > > We have also tried to use OpenSSL methods directly without using
> > the Dll
> > > but we get the same failure. This was also used with server and
> > client
> > > on the same machine as well as different machines with the same
> > > outcome. The non secure communication works fine between the
> > server and
> > > the client
> >
> > What does SSL_get_error() report after SSL_accept() fails?
> >
> > Also please dump the OpenSSL error stack when it fails, e.g. using
> > something like ERR_print_errors_fp(stdout);
> >
> > Matt
> >
>
