I have found the reason for the problem. Deep inside my program was a bug that switched modulus and exponent, so I set the wrong values to n and e. I've fixed the bug and also removed the leading 00 so that I get a good 128 byte long modulus and a 4 byte long exponent.
If I call RSA_verify I get now the reason 155 as error description. If I call RSA_public_decrypt I get a NULL pointer access violation. What can I do now? Could the NULL pointer exception be because of missing structure entries to RSA? I just do RSA_new() and the set only n and e. Frank Wockenfuß -----Ursprüngliche Nachricht----- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Wockenfuß, Frank Gesendet: Donnerstag, 2. August 2007 16:39 An: openssl-users@openssl.org Betreff: AW: AW: AW: AW: AW: RSA_public_decrypt problem Maybe the leadings 00 is the problem with both? Frank Wockenfuß -----Ursprüngliche Nachricht----- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Wockenfuß, Frank Gesendet: Donnerstag, 2. August 2007 16:36 An: openssl-users@openssl.org Betreff: AW: AW: AW: AW: AW: RSA_public_decrypt problem N: 008796FB4EAAB5FCC21619608ECB34D4BD82D062BF136A54E7E0BF6B2991C2F0F93A161930D650AF939C8282431D291D0E6E9F69A09AF091345D60439569C5CB5ECA566740B6A69FE4BBF2DB9CC03786AEDF8F9522EB7F6096A1B900140E6AA7AF55198B87E68A69546631E9EF90666984123F5364BE2EA6E067BBAA8831A34B15 E: 0040000081 Frank Wockenfuß -----Ursprüngliche Nachricht----- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Marek Marcola Gesendet: Donnerstag, 2. August 2007 15:39 An: openssl-users@openssl.org Betreff: Re: AW: AW: AW: AW: RSA_public_decrypt problem Hello, > RSA_NO_PADDING leads to the same error. Exactly, this fails on first stage. Maybe you can send (n,e) pair ? Best regards, -- Marek Marcola <[EMAIL PROTECTED]> ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]