Nevermind, I think I figured out my problem. Seems like the other crypto libs I'm using assumes the counter is in the lower 64 bits of the initialization vector. But OpenSSL looks like it was built with L_ENDIAN defined which assumes the counter is in the upper 64 bits. Does this sound right? After I copied the code from aes_ctr.c and modified the AES_ctr128_inc to remove the L_ENDIAN code, things seem to work as expected now. Can somebody comment on why this is? I saw some discussion on the openssl-dev list back in 2003. Is there no standard on where the counter should reside? If it is different for different libs, how do libs inter-operate with each other? Thanks, Ed
________________________________ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Edward Chan Sent: Friday, December 22, 2006 11:28 PM To: openssl-users@openssl.org Subject: Question on how to use AES-128-CTR I'm playing with various crypto libraries to encrypt/decrypt in AES-128-CTR. 2 of the 3 libs inter-operate fine with each other; i.e. I can encrypt with one and decrypt with the other and vice versa. However, when I use openssl with any of these libs, I am having problems. It seems like up to the first 16 bytes is decrypted fine. But after that, it's messed up. For example, if I use openssl to encrypt the string, "12345678901234567890", the other end will only decrypt "1234567890123456" followed by 4 bytes of garbage. And if the other lib encrypts "12345678901234567890", openssl will only decrypt "1234567890123456" followed by 4 bytes of garbage. However, if both ends is openssl, everything seems fine. But because the other 2 libs seem to work well with each other, I guess I'm not using the openssl api's correctly? Below is the code for my AES-128-CTR encrypter/decrypter class AES128CTR { protected: bool m_bEncrypt; // indicates if this is used to encrypt or decrypt AES_KEY m_key; U8 m_iv[AES_BLOCK_SIZE]; U8 m_ecount_buf[AES_BLOCK_SIZE]; U32 m_num; public: AES128CTR(const U8* key, U32 len, const U8* iv, U32 ivlen, bool bEncrypt) : m_bEncrypt(bEncrypt), m_num(0) { assert(len >= 16 && ivlen >= 16); // if (len < 16 || ivlen < 16) return; // key and iv need to be 128-bits len = ivlen = 16; memcpy(m_iv, iv, ivlen); memset(m_ecount_buf, 0, sizeof(m_ecount_buf)); int ret = AES_set_encrypt_key(key, len*8/*bits*/, &m_key); // returns 0 for success assert(!ret); } virtual ~AES128CTR() { } bool encrypt(const U8* in, U32 inlen, U8* out, U32* outlen) { if (m_bEncrypt) { AES_ctr128_encrypt(in, out, inlen, &m_key, m_iv, m_ecount_buf, &m_num); *outlen = inlen; return true; } return false; } bool decrypt(const U8* in, U32 inlen, U8* out, U32* outlen) { if (!m_bEncrypt) { // NOTE: calling AES_ctr128_encrypt to decrypt because AES_ctr128_encrypt is its own inverse. AES_ctr128_encrypt(in, out, inlen, &m_key, m_iv, m_ecount_buf, &m_num); *outlen = inlen; return true; } return false; } }; .... // this is how I encrypt data AES128CTR aesEncrypt; const char* str = "12345678901234567890"; U8 ciphertext[1024]; U32 len = sizeof(ciphertext); aesEncrypt.encrypt((U8*)str, strlen(str), ciphertext, &len); ... // this is how I decrypt data AES128CTR aesDecrypt; U8 plaintext[1024]; U32 len = sizeof(plaintext); aesDecrypt.decrypt(in, inlen, plaintext, &len);