Authentication failure between Non-FIPS SSHD and FIPS SSH

[Hai-May Chao]

According to the comments from 
http://www.mail-archive.com/openssl-users@openssl.org/msg58912.html 
<http://www.mail-archive.com/openssl-users@openssl.org/msg58912.html>,
We should use EVP_Signxxx/EVP_Verifyxxx in FIPS mode.
I verified that if both SSHD and SSH are running in FIPS mode, and both 
are using EVP_Signxxx/EVPVerifyxxx,
then the verifications were successful.

However, the following scenario would fail in the signature verification,

For Server authentication (assume RSA key pair is used) - at the end of 
Key Exchange

SSHD (running in Non-FIPS mode) uses RSA_sign() to sign the signature
SSH (running in FIPS mode) use EVP_Verifyxxx() to verify SSHD's 
signature ==> incorrect signature


For Client authentication (assume RSA key pair is used) - after entering 
the passphrase

SSH (running in FIPS mode) use EVP_Signxxx() to sign the signature
SSHD (running in Non-FIPS mode) uses RSA_public_decrypt() ==> incorrect 
signature (hash mismatch)


Is it the normal behavior with expecting verification error? And why is 
it occurring?

Thanks,
Hai-May