From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Bill Colvin Sent: Monday, 24 November, 2008 08:48
From is filled with random data. It is not a zero terminated text string, therefore, strlen(from) will probably be invalid. Well, partly filled; but the rest is uninitialized, and can be equally problematic -- although on most modern OSes, each process' bss space is initially zeroed and thus the first few heap allocations are very likely to get all-zeroes. ---------------------------------------------------------------------------- -- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of chamara caldera Sent: November 24, 2008 7:58 AM <snip> RSA *rsa = RSA_generate_key(1024, 3, NULL, NULL); unsigned char *from = (char *)malloc(1024); unsigned char *to = (char *)malloc(1024); unsigned char *from2 = (char *)malloc(1024); RAND_bytes(from, 32); int i = RSA_public_encrypt(strlen(from), from, to, rsa, RSA_PKCS1_OAEP_PADDING); printf("%d\n", i); i = RSA_private_decrypt(sizeof(to), to, from2, rsa, RSA_PKCS1_OAEP_PADDING); printf("%i\n",i); Also, sizeof(to) is the size of the pointer (4 bytes on most systems) not of the data. In general you should save the (nonerror) return from encryption and use that as size of the ciphertext; or in this case you know it's 128 bytes from the keysize. While I'm at it: You don't need 1024 BYTES for 'to' (ciphertext); for a 1024 bit RSA key the actual ciphertext will always be 128 bytes. (Although in most applications you wrap the actual ciphertext in something, like an ASN.1 encoding, that adds some bytes.) Similarly your cleartext can't be more than 128 bytes for such a key; it can be less, at least if using good padding, which you do with OAEP. You needn't cast the value returned by malloc -- in C void* will convert to any data pointer type -- and normally shouldn't as it can mask errors. You SHOULD #include <stdlib.h> to declare it, but rsa.h->crypto.h happens to do this for you. But your use of both decl-after-statement and implicit-int on main() is not compatible with either C standard (C89 or C99), or even C++. ERR_load_crypto_strings includes ERR_load_RSA_strings so that's redundant. You didn't seed the PRNG; depending on your system and how your openssl was built, you may not be getting good entropy in your key and data. For experimenting with the logic this doesn't matter, but if you wanted to use a program like this for anything serious, it would be an issue.
