Thanks Dr. Stephen and Victor for explanation
Some questions further in this :
Is there an API to configure programmatically , what hash algorithm it
can use. My understanding is that final ciphers which are selected are
for encryption and HMAC generation of application data packets. This
might be required to maintain the compatibility issues.
My understanding is that this error happens on client side, when it
tries to sign the CertificateVerify message, where digest output comes
too big for RSA to encrypt. Let me know if this is correct.
Thanks & Regards,
Nayna Jain
From: "Dr. Stephen Henson" <st...@openssl.org>
To: openssl-users@openssl.org
Date: 02/27/2013 04:26 AM
Subject: Re: What is the reason for error "SSL negotiation failed:
error:04075070:rsa routines:RSA_sign:digest too big for rsa
key"
Sent by: owner-openssl-us...@openssl.org
On Tue, Feb 26, 2013, Viktor Dukhovni wrote:
> On Wed, Feb 27, 2013 at 12:49:55AM +0530, Thulasi Goriparthi wrote:
>
> > Or use another hash type for signature which can produce not more than
53
> > bytes of hashed data. (i.e MD5, SHA1, SHA256, SHA384) while using
512-bit
> > keys. OpenSSL by default uses SHA512 hash for signature. Change the
code to
> > use any other hash.
>
> Interestingly enough, it is in fact SHA384 that fails with RSA-512. The
> client and server agree on:
>
> ECDHE-RSA-AES256-GCM-SHA384
>
> > 512 bit(64 byte) RSA key can only encrypt 53 bytes at max. 64 - 11 byte
> > padding and SHA512 produces 64 bytes of hashed data.
>
> and the handshake fails when the client's key is RSA-512. Indeed
> the shortest RSA key that seems to work is RSA-745, tests with
> RSA-744 consistently fail. I don't know why the requisite key size
> is substantially larger than the digest length + expected padding.
>
The signature format for TLS 1.2 is the standard DigestInfo structures. For
SHA2 algorithms this adds an additional 19 bytes of structure before the
digest itself.
In (unreleased) OpenSSL 1.0.2 and later the signature algorithms used by
client authentication can be set by the application. For 1.0.1 it just
includes
everything supported by the library and selects the first preference for
the
key type specified. That means using OpenSSL 1.0.1 on both ends will use
SHA512 which is fine for everything but insecure key sizes with RSA.
It could be fixed to handle the insecure key sizes too but do we really
want
people to use those?
Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
