Using the EVP_Signxxx API to perform a FIPS mode DSA key sign will generate a signature with ASN.1 format (fips_dsa_sign.c). Therefore, the signature size is no longer 40 bytes (rlen = 20 plus slen = 20) but 48 bytes with padding and ASN.1 overhead (rlen = 20, rpad = 1, slen = 20, spad = 1, SEQUENCE = 4, SEQUENCE header = 2). There are two potential issues for this:
1) SSH always assume the DSA signature is 40 bytes. This will cause the sigblob[SIGBLOB_LEN] (in ssh-dss.c) to be overflowed by a DSA signature in ASN.1 format, and thus smashes the SSH's stack. 2) When a SSH client in FIPS mode communicates with a non-FIPS SSHD using the DSA key authentication. the authentication will fail. This is because SSH will generate a 48 bytes signature (in ASN.1 format) by calling the EVP_SignFinal() API, while the SSHD will still assume a 40-byte raw signature when performing the verification. Is there a way to resolve these issues (specially on issue #2)? Thank you very much. Hai-May ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
