I found the solution for the interoperability problem between Java and C: no padding does not work, but PKCS1PADDING does.
In the Java code below, change the line: Cipher c = Cipher.getInstance("RSA/ECB/PKCS1PADDING"); And in the C code the change at decryption is to use RSA_PKCS1_PADDING instead of RSA_NO_PADDING: Regards, Paul I wrote earlier: > 22.08.2008 17:09 > > Hi, > > we have a server using OpenSLL crypto library functions around RSA. > I want to write a Java client that sends an encrypted message to the > server, which can be decrypted there. > > To test this I use the following Java code below: > > The encoded public key is created with OpenSSL. > RSA* rsa_private= RSA_generate_key (keylength, RSA_F4, NULL, NULL); > len = i2d_RSA_PUBKEY(rsa_private, NULL); > pBuffer=(char *)OPENSSL_malloc(len); > upBuffer = (unsigned char*)pBuffer; > len = i2d_RSA_PUBKEY(rsa_private, &upBuffer); > > The server decrypts the password with the same private key used to > generate the public key: > retval = RSA_private_decrypt(RSA_size(rsa_private), > encryptedPasswordCharArrayfromJavaClient, cleartext-output, > rsa_private, RSA_NO_PADDING); > > I tested my code so that encryption / decryption works in the C > code using OpenSSL. > I tested my Java code so that encryption / decryption works in the > Java code using standard Java packages with a Sun provider for RSA. > I have trouble to get the two together. If I use RSA_private_decrypt > with a byte sequence created from the Java client the output is > wrong - no error occurs. > > One thing that might point to a reason for the incompatibility: > I observe that the Java client always creates the same encrypted > byte sequence from a given input string and a public key. > The OpenSSL C client produces different encrypted byte sequences for > each separate run with a given input string and a public key. > Decryption works with all of them. > If I let the C client encrypt the same given input string twice in > the same run, it creates the same encrypted sequence. > > The java client code: > > import java.security.KeyFactory; > import java.security.PublicKey; > import java.security.spec.X509EncodedKeySpec; > import java.util.Date; > > import javax.crypto.Cipher; > > public class ClientX509Encryption { > > static final byte[] encodedPublicKey = { > 48, -127, -97, 48, 13, 6, 9, 42, -122, 72, -122, -9, 13, 1, 1, 1, > 5, 0, 3, -127, -115, 0, 48, -127, -119, 2, -127, -127, 0, -77, 119, -61, > 76, -66, -58, -102, -98, 84, 20, 83, 117, -9, 69, 109, -105, 51, -58, 12, > 31, -113, 110, -118, 20, 115, 107, -86, -97, 14, -39, 62, 8, 50, -22, 65, > 121, -124, -52, 27, 17, -104, -123, -78, -56, 92, 18, -31, 116, > 69, 105, -96, > -87, 44, 60, -115, -110, 8, -124, 59, -68, 54, 33, 63, 108, 24,114, -11, > -103, -31, 48, -89, 10, 98, 106, -80, 123, -94, -100, -24, -59, > 18, 112, 74, > 112, 58, -94, -23, 70, -41, 35, 92, 92, 26, -27, -51, 127, > -125, -109, 15, > -30, -18, -12, 27, 101, 92, 4, 77, 11, -22, 123, 47, -18, 88, -96, 95, > 32, -118, -128, 82, 22, -55, -39, 9, -34, 4, 90, 88, -17, 2, 3, 1, > 0, 1, > }; > > public static void main(String[] args) { > > byte[] encryptedpassword = null; > > try { > Cipher c = Cipher.getInstance("RSA/ECB/NoPadding"); > > X509EncodedKeySpec publicKeySpec = new X509EncodedKeySpec(encodedPublicKey > ); > > KeyFactory keyFactory = KeyFactory.getInstance("RSA"); > PublicKey publicKey = keyFactory.generatePublic(publicKeySpec); > > c.init(Cipher.ENCRYPT_MODE, publicKey); > > byte[] arg0Bytes = args[0].getBytes(); > > // encrypt input parameter > encryptedpassword = c.doFinal(arg0Bytes); > > System.out.print("static signed char > encryptedPasswordStringfromJavaClient[] = {"); > for (int i = 0; i < encryptedpassword.length; i++) { > if (i % 16 == 0) { > System.out.println(""); > System.out.print(" "); > } > System.out.print(encryptedpassword[i] + ", "); > } > System.out.println("\n};"); > > } catch (Exception e) { > e.printStackTrace(); > } > > } > }