> Though the RSA_sign function signature doesn't seem to allow specifying
the padding.
Sorry I meant the hashing algorithm.
So I went back to this and I think the problem is we are forced to
create a RSA_private_encrypt function even if all we want to do is sign.
https://github.com/openssl/openssl/blob/master/crypto/rsa/rsa_pmeth.c#L184
That branch gets hit on doing a TLS connection but not on creating a
certificat
Operations that a private key can do are decrypt and sign. Similarly,
operations that a public key can do are encrypt and verify.
The legacy priv_enc(raw) just refers to raw signing, and is almost same as
sign(with proper padding mechanisms).
It is just a misnomer, as data encrypted with a privat
I implemented some custom engines and RSA_meth_set_priv_enc seems to map
to other libraries' RSA decrypt operation (NCryptDecrypt on Windows
CNG, Cipher class with Cipher.DECRYPT_MODE on Android). They can do a
TLS connection just fine with a self-signed cert.
I looked at another custom e
