I am trying to run wpa_supplicant in FIPS mode. The stack trace of the
failure is below. I'm assuming that an algorithm not supported by FIPS
is being used but I'm not sure which one since I thought I removed
them all. I replaced SHA-1 with
EVP_MD_CTX ctx;
size_t i;
unsigned int mac_len;
EVP_DigestInit(&ctx, EVP_sha1());
for(i = 0; i < num_elem; i++)
{
EVP_DigestUpdate(&ctx, addr[i], len[i]);
}
EVP_DigestFinal(&ctx, mac, &mac_len);
and MD5 with
MD5_CTX ctx;
size_t i;
MD5_Init(&ctx);
for(i = 0; i < num_elem; i++)
{
MD5_Update(&ctx, addr[i], len[i]);
}
MD5_Final(mac, &ctx);
which I thought allowed me to bypass the FIPS check. I don't think the MD5
function is even called so I want to know at least if there is a way to
determine what cipher is being called that is failing and any additional
clues would be also appreciated.
digest.c(151): OpenSSL internal error, assertion failed: Digest update
previous FIPS forbidden algorithm error ignored
Breakpoint 1, 0x401d48fc in abort () from /lib/libc.so.6
(gdb) bt
#0 0x401d48fc in abort () from /lib/libc.so.6
#1 0x0001c088 in OpenSSLDie ()
#2 0x0001f970 in bad_update ()
#3 0x0001f830 in EVP_DigestUpdate ()
#4 0x4003ffa8 in ssl3_finish_mac () from /usr/lib/libssl.so.0.9.8
#5 0x400432e0 in ssl3_do_write () from /usr/lib/libssl.so.0.9.8
#6 0x4003e6b8 in ssl3_connect () from /usr/lib/libssl.so.0.9.8
#7 0x4004ebf4 in SSL_connect () from /usr/lib/libssl.so.0.9.8
#8 0x000526c0 in tls_connection_handshake (ssl_ctx=<value optimized
out>, conn=0x94e68, in_data=0x970d0 "\224,@\224,@\020", in_len=0,
out_len=0x94d88, appl_data=0xbed1f218, appl_data_len=0xbed1f214) at
../src/crypto/tls_openssl.c:1792
#9 0x0004fcac in eap_peer_tls_process_helper (sm=0x8ad90,
data=0x94d80, eap_type=EAP_TYPE_PEAP, peap_version=0, id=98 'b',
in_data=0x0, in_len=537439632, out_data=0xbed1f264) at
../src/eap_peer/eap_tls_common.c:463
#10 0x00049354 in eap_peap_process (sm=0x8ad90, priv=0x94d80,
ret=0xbed1f298, reqData=0x94d68) at ../src/eap_peer/eap_peap.c:1076
#11 0x0004dcf4 in sm_EAP_METHOD_Enter (sm=0x8ad90, global=<value
optimized out>) at ../src/eap_peer/eap.c:368
#12 0x0004e22c in eap_peer_sm_step (sm=0x8ad90) at ../src/eap_peer/eap.c:698
#13 0x0004b9d0 in eapol_sm_step (sm=0x8ac68) at
../src/eapol_supp/eapol_supp_sm.c:900
