Hi!
 
We are trying to provide our own callbacks to the RSA method thus overriding the defaults ( using engine version of openssl).
 
Simplified version of existing code (that does not use engine openssl)
 
void setCustomCallback(SSL *ssl, MyCtxt *myCtxt)
{
 RSA *rsa;
 RSA_METHOD *meth;
 X509* myCert;
 int ret;
 EVP_PKEY *pKey=NULL;
 
 if ((myCert = createCert(derCert, derCertLen)) == NULL)
  return -1;
 
 rsa = RSA_new();
 
 meth = (RSA_METHOD*) malloc(sizeof(RSA_METHOD)); 
 *meth = *rsa->meth; /* ANSI struct assignment */
 
 /* Set custom callbacks */
 meth->rsa_priv_dec = (int(*)())capiRsaPrivDec;
 meth->rsa_priv_enc = (int(*)())capiRsaPrivEnc;
 meth->finish = (int(*)())capiRsaFinish;
 
 meth->app_data = (char*) myCtxt;
 meth->flags |= RSA_METHOD_FLAG_NO_CHECK;
 rsa->meth = meth;
 

 if ((pKey = X509_get_pubkey(myCert)) == NULL)
  return -1;
 
 rsa->n = BN_dup(pKey->pkey.rsa->n);
 rsa->e = BN_dup(pKey->pkey.rsa->e);
 
 ret = SSL_use_RSAPrivateKey(ssl, rsa);
 
 RSA_free(rsa);
 
 if (ret != 1)
 {
  return -1;
 }
 
 if (SSL_use_certificate(ssl, myCert) != 1)
  ret = -1;
 else
  ret = 0;
 
 X509_free(myCert);
}
 
How do we re-write this to use the engine version?
 
Issues:
 
 Method data member is replaced by engine_st in the Openssl-engine
version so
rsa->meth = meth is invalid.
 
the nearest call equivalent of this is
RSA_set_method (RSA *pRsa, ENGINE *pEngine)
 
The way RSA_set_method seems to work is
a. From pRsa get old engine reference and save a reference to
old engine
b. From old engine reference get the method
c. Init new pEngine
d. if finish cb defined in old method execute finish_cb
e. Assign the new pEngine to pRsa
f. If new pEngine method has init cb defined execute init_cb
g. Using the old engine referenc do ENGINE_finish
 
From the above it looks like we need to clone a Engine from
the existing old engine.
 
In this new cloned structure assign the callbacks for the RSA METHOD
and now call RSA_set_method with this newly created Engine.
 
The question now is how do I create a clone of an existing engine in
proper way.
 
Thanks
Ramesh AV

Reply via email to