Problems with RSA encryption/decryption

[Dan Spirlock]

Hello,

Extreme noob to openssl, and am running into problems using the  
crypto library for RSA encryption and decryption.  For now I'm just  
putting together a simplistic test application to get a feel for  
using the openssl libs.   The problem I'm having is when I try to  
decrypt an encrypted message using a private key, I get two errors  
that I don't know how to begin debugging:

RSA_padding_check_PKCS1_OAEP:oaep decoding error:

and

RSA_EAY_PRIVATE_DECRYPT:padding check failed

I'm trying to use code pulled more or less from the examples in the  
"Network Security with OpenSSL" OReilly book.  I'm wondering if  
someone can let me know what might be causing the errors, and if I'm  
even on the right track with my approach.

Thanks in advance for any responses,
Dan

Here is the short test application I've written:


#include <cstdio>
#include "openssl/rsa.h"
#include "openssl/pem.h"
#include "openssl/err.h"

using namespace std;
int main ()
{
        RSA     *rsaPrivKey = RSA_new();
        RSA     *rsaPubKey = RSA_new();
        unsigned char inText[512];
        
        ERR_load_crypto_strings();
        FILE  *pubKey = fopen("rsa_publickey.pem", "r");
        FILE  *privKey = fopen("rsa_privatekey.pem", "r");
        if(pubKey)
        {
                rsaPubKey = PEM_read_RSA_PUBKEY(pubKey, &rsaPubKey, NULL, NULL);
        }
        if(privKey)
                rsaPrivKey = PEM_read_RSAPrivateKey(privKey, &rsaPrivKey, NULL, 
NULL);

        cout << "Enter a value to be encoded: " << endl;
        cin >> inText;

        // Encryption
        int sigretVal = RSA_size(rsaPubKey);
        unsigned char *sigBuffer = new unsigned char[sigretVal];
        int size = strlen((char*)inText);
	RSA_public_encrypt(size, inText, sigBuffer, rsaPubKey,  
RSA_PKCS1_OAEP_PADDING);

        // Decryption
        sigretVal = RSA_size(rsaPrivKey);
        unsigned char *plainText = new unsigned char[sigretVal];
        int encSize = strlen((char*)sigBuffer);
        cout << "Encoded data size: " << encSize << endl;
	RSA_private_decrypt(encSize, sigBuffer, plainText, rsaPrivKey,  
RSA_PKCS1_OAEP_PADDING);
        ERR_print_errors_fp(stdout);
        cout << "The decrypted message is: " << plainText << endl;

        RSA_free(rsaPubKey);
        RSA_free(rsaPrivKey);
        delete [] sigBuffer;
        delete [] plainText;

}