Hi,
I'm trying to create a RSA signature for a message. Here's the rough
code:
-----
hash = (unsigned char *)malloc(RSA_size(rsa) * sizeof(unsigned char));
ciphertext = (char *)malloc(RSA_size(rsa) * sizeof(char));
signature = (char *)malloc(RSA_size(rsa) * sizeof(char));
if (ciphertext != NULL && signature != NULL && hash != NULL) {
memset(ciphertext, 0, RSA_size(rsa));
ok = RSA_private_encrypt(strlen(reply), (unsigned char *)reply,
(unsigned char *)ciphertext, rsa, RSA_PKCS1_PADDING);
if (ok < 0) {
derror1("Message encryption error: %s",
ERR_error_string(ERR_get_error(), (char
*)NULL)); return (true);
} else {
dtrace1("Message encryption successful; return value:
%d", ok); }
len = base64Encode(ciphertext, ok);
memset(hash, 0, RSA_size(rsa));
RIPEMD160((unsigned char *)ciphertext, len, hash);
memset(signature, 0, RSA_size(rsa));
ok = RSA_private_encrypt(RIPEMD160_DIGEST_LENGTH, hash,
(unsigned char *)signature, rsa, RSA_PKCS1_PADDING);
-----
That final line causes a segmentation fault. Here is a backtrace:
-----
#0 0x00007ffff74d0ba6 in ?? () from /lib/libc.so.6
#1 0x00007ffff74d2aa0 in malloc () from /lib/libc.so.6
#2 0x00007ffff7abc962 in CRYPTO_malloc (num=-142946720,
file=0x7ffff7b7f5a0 "bn_ctx.c", line=409) at mem.c:328 #3
0x00007ffff7ae3d17 in BN_POOL_get (ctx=0x62b680) at bn_ctx.c:409
#4 BN_CTX_get (ctx=0x62b680) at bn_ctx.c:293
#5 0x00007ffff7b012c6 in RSA_eay_private_encrypt (flen=20, from=<value
optimized out>, to=<value optimized out>, rsa=0x617010, padding=1) at
rsa_eay.c:358
#6 0x000000000040bd80 in handleCommunication (commSock=6,
confFile=0x7fffffffe010 "/etc/activation.conf", clientAddr=...) at
activation.c:234
#7 0x000000000040c6bf in main (argc=1, argv=0x7fffffffe518) at
activation.c:421
-----
The funny thing is that it *mostly* crashes at this point, but it
occasionally goes well. I've been able to reproduce this problem on
various machines, though, so I think I can rule out a problem with
faulty RAM.
Same behaviour when using the RSA_sign function instead.
After hours of debugging, I'm running out of ideas. What could be
causing this?
Hannes
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
