Sorry for the length of this post -- if you do not write software, you can probably hit the delete key now.
I would like to express my thanks for all those that provided input regarding my compatibility issue between AES_cbc_encrypt() and the Perl Crypt::CBC / Crypt::OpenSSL::AES implementation. Based on comments I received from several people, I was able to narrow the problem down and resolve the issue. The problem was that I did not properly pad my input data, and my length calculations were incorrect as a result. To help others that may encounter problems, I have summarized what I learned, and have provided working code below that can be used by developers as a starting point. If you are working with AES for the first time, there are some basic things in this list that you must understand before you begin to write code. 1. The OpenSSL routines, specifically AES_cbc_encrypt(), require all input lengths to be an even multiple of the block size. For AES, the block size is always 16. To align the data into even blocks, use padding. 2. The AES_cbc_encrypt() function will not add padding when encrypting, and will not remove padding when decrypting. Padding is the responsibility of the programmer. 3. If you want interoperability with other AES implementations, the most compatible form of padding is "standard padding" as defined in PKCS#5. This is easy to implement, and code for this is provided at the end of this post in the PadData() function. Essentially, the padding value is equal to the number of bytes required to make the data a multiple of the block size (16). If the data is already an even multiple of block size, you must add an extra block of padding. Here are two examples: Example 1: Block size is 16, data is 25 bytes long. To get to an even multiple of block size, data must be padded to 32 bytes. The pad character should be hex "07" (32 - 25). Example 2: Block size is 16, data is 16 bytes long. The data is already an even multiple of 16, however we still must account for padding, so we add and extra block of padding to make the total length 32 bytes. The padding character should be hex "10" (32 - 16). 4. To remove padding, all you need is to look at the value of the last character in the decrypted result and subtract that many characters from the end of the array. I provide a function below that shows how to do this named NoPadLen(). 5. The length of the output data created by the AES_cbc_encrypt() function is the same as the input data. There is a one for one input to output ratio for encryption and decryption. 6. You need both the key and the initialization vector to encode and decrypt the data. The initialization vector is changed when AES_cbc_encrypt() returns, so if you intend to encrypt and decrypt in the same pass, you must reset the the initialization vector before your call to decrypt the data. The initialization vector is one block size in length (16 bytes for AES). 7. When using Crypt::CBC in Perl, there are several things you need to understand in order to be interoperable with OpenSSL AES encryption. These items are listed below: a. Crypt::CBC takes the key provided to it and computes an MD5 hash of the value in order to yield a 256 bit key for AES. To get Crypt::CBC to use an actual key, you must override that "feature" by telling Crypt::CBC to use a literal key with the "literal_key => 1" parameter. Note: This is "literal_key", not "literal-key". :) b. Crypt::CBC assumes a salt header, which is incompatible with AES_cbc_encrypt() (unless you want to compute your own salt headers). To disable the salt header in Crypt::CBC, use the "-header => "none"" parameter. c. You must specify the initialization vector with Crypt::CBC. This is done with the "-iv => "YourIvPlacedHere"" parameter. The same initialization vector must be used for encrypting and decrypting. The IV must be 16 bytes long. d. The encryption method for AES under Crypt::CBC must be "Crypt::OpenSSL::AES". e. This is an example of a request to create an OpenSSL compatible cipher object using the Crypt::CBC module in Perl: my $obj = Crypt::CBC->new ( -key => "abcdefghijklmnopqrstuvwxyz012345", -literal_key => 1, -header => "none", -iv => "RandomIVRandomIV", -cipher => "Crypt::OpenSSL::AES" ); Code Examples: I have provided six example console applications for working with AES in both C and Perl. Note that the C code was developed under Microsoft Windows Visual C and uses the secure forms of gets(), memcpy(), and sprintf() -- you will need to perform the appropriate substitutions for these if you port this to UNIX. The programs are: aesenc.c : encrypt user input into hex coded string in C aesdec.c : decrypt hex coded string into source input in C aesencdec.c : loop to demonstrate AES encrypt and decrypt in C aesenc.pl : encrypt user input into hex coded string in Perl aesdec.pl : decrypt hex coded string into source input in Perl aesencdec.pl : loop to demonstrate AES encrypt and decrypt in Perl You can take the output from aesenc.c and decode it with aesdec.pl to demonstrate compatibility between the two platforms. The same is true for aesenc.pl and aesdec.c. The source code for these programs is provided below. =================================================================== /* * aesenc.pl - encrypt data and hex encode using AES CBC with a 256 bit key * * Dave Stoddard - 12/11/2008 * dgs at accelix.com * * This software is placed the public domain and free for any use. * No warrantees. Use at your own risk. * * Description * ----------- * This is a demonstration program for illustrating interoperability between * C and Perl using AES CBC encryption. The program functions as a console * application that prompts the user for input, encrypts and hex encodes the * input, and prints the encrypted result. The result may be decrypted using * aesdec.c or aesdec.pl. To exit the loop, enter "exit" or "quit" at the * prompt. */ # include <stdio.h> # include <ctype.h> # include <string.h> # include "openssl/aes.h" # define BLOCK_LEN 16 # define MAXBUF 65536 # define MAXHEX (MAXBUF * 2) + 1 # define KEY "abcdefghijklmnopqrstuvwxyz012345" # define IV "RandomIVRandomIV" // function prototypes unsigned int bin2hex (unsigned char *in, unsigned char *out, unsigned int len); unsigned int PadData (unsigned char *ibuf, unsigned int ilen, int blksize); int main(int argc, char *argv[]) { unsigned long ilen; unsigned char data[MAXBUF]; // command line input unsigned char ibuf[MAXBUF]; // encrypt input unsigned char obuf[MAXBUF]; // encrypt output unsigned char xbuf[MAXHEX]; // hex encrypt output unsigned char key[] = KEY; unsigned char iv[] = IV; AES_KEY aeskeyEnc; // loop through user input while (1) { // read input printf ("input> "); gets_s (data,MAXBUF); // check for exit if (strcmp (data,"exit") == 0) break; if (strcmp (data,"quit") == 0) break; // prepare the input data with padding memset (ibuf, 0x00, sizeof (ibuf)); memcpy_s (ibuf, sizeof (ibuf), data, strlen (data)); // calc length of aes output block ilen = PadData (ibuf, (unsigned int) strlen (data), BLOCK_LEN); // init cipher keys AES_set_encrypt_key (key, 256, &aeskeyEnc); // encrypt string memcpy_s (iv, sizeof (iv), IV, sizeof (IV)); AES_cbc_encrypt (ibuf, obuf, ilen, &aeskeyEnc, iv, AES_ENCRYPT); // convert encoded string to hex and display bin2hex (obuf,xbuf,ilen); printf ("encode: %s (len = %d)\n", xbuf, (int) strlen (xbuf)); // add blank line for input prompt printf ("\n"); } // while (1) return (0); } /* * bin2hex() takes a block of data provided in pcIbuf with a length * of len and converts it into a zero-terminated ascii string of * hexadecimal characters. It returns the length of the resulting * hexadecimal string (exclusive of the terminating zero). */ unsigned int bin2hex (unsigned char *pcIbuf, unsigned char *pszObuf, unsigned int ilen) { unsigned int i; // loop iteration conuter unsigned int j = (ilen * 2) + 1; // output buffer length unsigned char *p; p = pszObuf; // point to start of output buffer for (i = 0; i < ilen; i++) { sprintf_s (p, j, "%2.2x", (unsigned char) pcIbuf [i]); p += 2; j -= 2; } *p = '\0'; return (0); } /* * The PadData() function is used to pad input data * for cipher block chaining using standard padding * as specified in PKCS 5. Input data is padded in * place. ilen is the length of the data before * padding. blksize is the block size to use (16 for * AES CBC). The function returns the length of ibuf * after padding. Note that ibuf must be able to hold * at least blksize extra bytes. */ unsigned int PadData (unsigned char *ibuf, unsigned int ilen, int blksize) { unsigned int i; // loop counter unsigned char pad; // pad character (calculated) unsigned char *p; // pointer to end of data // calculate pad character pad = (unsigned char) (blksize - (ilen % blksize)); // append pad to end of string p = ibuf + ilen; for (i = 0; i < (int) pad; i++) { *p = pad; ++p; } return (ilen + pad); } // end aesenc.c =================================================================== /* * aesdec.c - decrypt hex encoded strings using AES CBC with a 256 bit key * * Dave Stoddard - 12/11/2008 * dgs at accelix.com * * This software is placed the public domain and free for any use. * No warrantees. Use at your own risk. * * Description * ----------- * This is a demonstration program for illustrating interoperability between * C and Perl using AES CBC encryption. The program functions as a console * application that prompts the user for input, decodes and decrypts the input, * and prints the decoded result. To exit the loop, enter "exit" or "quit" * at the prompt. * * NOTE: The key and IV used to decode the input must be the same as the * encoding program (see aesenc.c or aesenc.pl). */ # include <stdio.h> # include <ctype.h> # include <string.h> # include "openssl/aes.h" # define BLOCK_LEN 16 # define MAXBUF 65536 # define MAXHEX (MAXBUF * 2) + 1 # define KEY "abcdefghijklmnopqrstuvwxyz012345" # define IV "RandomIVRandomIV" // function prototypes unsigned int hex2bin (unsigned char *in, unsigned char *out, unsigned int len); unsigned int NoPadLen (unsigned char *ibuf, unsigned int ilen); int main(int argc, char *argv[]) { unsigned long ilen; unsigned char data[MAXBUF]; // command line hex input unsigned char ibuf[MAXBUF]; // hex decrypt output unsigned char obuf[MAXBUF]; // decrypt output unsigned char key[] = KEY; unsigned char iv[] = IV; AES_KEY aeskeyDec; // loop through user input while (1) { // read input and convert \n to \0 printf ("encrypted hex input> "); gets_s (data,MAXBUF); // check for exit if (strcmp (data,"exit") == 0) break; if (strcmp (data,"quit") == 0) break; // init cipher keys AES_set_decrypt_key (key, 256, &aeskeyDec); // convert hex string to binary ilen = hex2bin (data, ibuf, (int) strlen(data)); // decrypt text string memcpy_s (iv, sizeof (iv), IV, sizeof (IV)); AES_cbc_encrypt (ibuf, obuf, ilen, &aeskeyDec, iv, AES_DECRYPT); obuf [NoPadLen (obuf, ilen)] = 0x00; printf ("decrypted: %s (len = %d)\n", obuf, strlen(obuf)); // add blank line for input prompt printf ("\n"); } // while (1) return (0); } /* * hex2bin() converts an array of hex digits into an array of binary values. * Inputs are pointers to the input and output character arrays, and the * length on the input string. The function returns the length of the * resulting output array, or 0 if there is an error. If the input string * is zero terminated, you can call this as follows: * * ilen = hex2bin (ibuf, obuf, strlen (ibuf)); * * Note that the output is not terminated by a zero as the output * list is likely contain binary data. */ unsigned int hex2bin (unsigned char *ibuf, unsigned char *obuf, unsigned int ilen) { unsigned int i; // loop iteration variable unsigned int j; // current character unsigned int by = 0; // byte value for conversion unsigned char ch; // current character // process the list of characaters for (i = 0; i < ilen; i++) { ch = toupper(*ibuf++); // get next uppercase character // do the conversion if(ch >= '0' && ch <= '9') by = (by << 4) + ch - '0'; else if(ch >= 'A' && ch <= 'F') by = (by << 4) + ch - 'A' + 10; else { // error if not hexadecimal memcpy (obuf,"ERROR",5); return 0; } // store a byte for each pair of hexadecimal digits if (i & 1) { j = ((i + 1) / 2) - 1; obuf [j] = by & 0xff; } } return (j+1); } /* * NoPadLen() will recalculate the length of an array of data after the pad * characters have been removed and will return the new length to the caller. * Note that nothing is altered by this routine -- it simply returns the * adjusted length after taking into account the removal of the padding. * * NOTE: Assuming "buf" is a character array that contains your data after * it has been decrypted (with padding still appended), you can remove the * padding and zero terminate the text string by using the following * construct: * * buf [NoPadLen (buf, len)] = 0x00; */ unsigned int NoPadLen (unsigned char *ibuf, unsigned int ilen) { unsigned int i; // adjusted length unsigned char *p; // pointer to last character p = ibuf + (ilen - 1); i = ilen - (unsigned int) *p; return (i); } // end aesdec.c =================================================================== /* * aesencdec.c - encrypt and decrypt data using AES CBC with a 256 bit key * * Dave Stoddard - 12/11/2008 * dgs at accelix.com * * This software is placed the public domain and free for any use. * No warrantees. Use at your own risk. * * Description * ----------- * This is a demonstration program for illustrating the encryption and * decryption of input data in C using AES CBC encryption that is * interoperable with Perl language programming using Crypt::CBC. * Refer to aesencdec.pl for an equivalent of this program in Perl. * * The program functions as a console application that prompts for user input, * encrypts and displays the hex encoded result, then decrypts the result and * displays the original input. To exit the input loop, enter "exit" or * "quit" at the prompt. */ # include <stdio.h> # include <ctype.h> # include <string.h> # include "openssl/aes.h" # define BLOCK_LEN 16 # define MAXBUF 65536 # define MAXHEX (MAXBUF * 2) + 1 # define KEY "abcdefghijklmnopqrstuvwxyz012345" # define IV "RandomIVRandomIV" // function prototypes unsigned int bin2hex (unsigned char *in, unsigned char *out, unsigned int len); unsigned int hex2bin (unsigned char *in, unsigned char *out, unsigned int len); unsigned int PadData (unsigned char *ibuf, unsigned int ilen, int blksize); unsigned int NoPadLen (unsigned char *ibuf, unsigned int ilen); int main(int argc, char *argv[]) { unsigned long ilen; unsigned char data[MAXBUF]; // command line input unsigned char ibuf[MAXBUF]; // padded input to encrypt unsigned char obuf[MAXHEX]; // encrypt output unsigned char xbuf[MAXHEX]; // hex encrypt output unsigned char ybuf[MAXBUF]; // hex decrypt output unsigned char dbuf[MAXBUF]; // decrypt output unsigned char key[] = KEY; unsigned char iv[] = IV; AES_KEY aeskeyEnc, aeskeyDec; // loop through user input while (1) { // read input and convert \n to \0 printf ("input> "); gets_s (data,MAXBUF); // check for exit if (strcmp (data,"exit") == 0) break; if (strcmp (data,"quit") == 0) break; // prepare the input data with padding memset (ibuf, 0x00, sizeof (ibuf)); memcpy_s (ibuf, sizeof (ibuf), data, strlen (data)); // pad and calc length of block to encode ilen = PadData (ibuf, (unsigned int) strlen (data), BLOCK_LEN); // init cipher keys AES_set_encrypt_key (key, 256, &aeskeyEnc); AES_set_decrypt_key (key, 256, &aeskeyDec); // encrypt string memcpy_s (iv, sizeof (iv), IV, sizeof (IV)); AES_cbc_encrypt (ibuf, obuf, ilen, &aeskeyEnc, iv, AES_ENCRYPT); // convert encoded string to hex and display bin2hex (obuf,xbuf,ilen); printf ("encode: %s (len = %d)\n", xbuf, (int) strlen (xbuf)); // convert hex string to binary hex2bin (xbuf, ybuf, (int) strlen(xbuf)); // compare binary values if (memcmp (obuf, ybuf, ilen) != 0) { printf ("HEX DECODE FAILED!\n"); return (0); } // decrypt string memcpy_s (iv, sizeof (iv), IV, sizeof (IV)); AES_cbc_encrypt (ybuf, dbuf, ilen, &aeskeyDec, iv, AES_DECRYPT); dbuf [NoPadLen (dbuf, ilen)] = 0x00; printf ("decode: %s (len = %d)\n", dbuf, strlen(dbuf)); // compare binary values if (memcmp (data, dbuf, (unsigned int) strlen (data)) != 0) { printf ("DECRYPT FAILED!\n"); return (0); } // insert blank line before next input prompt printf ("\n"); } // while (1) return (0); } /* * bin2hex() takes a block of data provided in ibuf with a length * of ilen and converts it into a zero-terminated ascii string of * hexadecimal characters. It returns the length of the resulting * hexadecimal string (exclusive of the terminating zero). NOTE: the * output buffer (obuf) must be able to accommodate twice the amount * of data as the input buffer + 1 byte for the zero terminator. */ unsigned int bin2hex (unsigned char *ibuf, unsigned char *obuf, unsigned int ilen) { unsigned int i; // loop iteration conuter unsigned int j = (ilen * 2) + 1; // output buffer length unsigned char *p; p = obuf; // point to start of output buffer for (i = 0; i < ilen; i++) { sprintf_s (p, j, "%2.2x", (unsigned char) ibuf [i]); p += 2; j -= 2; } *p = '\0'; return (strlen (obuf)); } /* * hex2bin() converts an array of hex digits into an array of binary values. * Inputs are pointers to the input and output character arrays, and the * length on the input string. The function returns the length of the * resulting output array, or 0 if there is an error. If the input string * is zero terminated, you can call this as follows: * * ilen = hex2bin (ibuf, obuf, strlen (ibuf)); * * Note that the output is not terminated by a zero as the output * list is likely contain binary data. */ unsigned int hex2bin (unsigned char *ibuf, unsigned char *obuf, unsigned int ilen) { unsigned int i; // loop iteration variable unsigned int j; // current character unsigned int by = 0; // byte value for conversion unsigned char ch; // current character // process the list of characaters for (i = 0; i < ilen; i++) { ch = toupper(*ibuf++); // get next uppercase character // do the conversion if(ch >= '0' && ch <= '9') by = (by << 4) + ch - '0'; else if(ch >= 'A' && ch <= 'F') by = (by << 4) + ch - 'A' + 10; else { // error if not hexadecimal memcpy (obuf,"ERROR",5); return 0; } // store a byte for each pair of hexadecimal digits if (i & 1) { j = ((i + 1) / 2) - 1; obuf [j] = by & 0xff; } } return (j+1); } /* * The PadData() function is used to pad input data for cipher block * chaining using standard padding as specified in PKCS#5. Input data * is padded in place. The ilen value is the length of the data before * padding. The blksize parameter is the block size to use (16 for AES CBC). * The function returns the new length of ibuf after padding. * * NOTE: The ibuf must be large enough to hold at least blksize additional * bytes of data to accommodate padding. */ unsigned int PadData (unsigned char *ibuf, unsigned int ilen, int blksize) { unsigned int i; // loop counter unsigned char pad; // pad character (calculated) unsigned char *p; // pointer to end of data // calculate pad character pad = (unsigned char) (blksize - (ilen % blksize)); // append pad to end of string p = ibuf + ilen; for (i = 0; i < (int) pad; i++) { *p = pad; ++p; } return (ilen + pad); } /* * NoPadLen() will recalculate the length of an array of data after the pad * characters have been removed and will return the new length to the caller. * Note that nothing is altered by this routine -- it simply returns the * adjusted length after taking into account the removal of the padding. * * NOTE: Assuming "buf" is a character array that contains your data after * it has been decrypted (with padding still appended), you can remove the * padding and zero terminate the text string by using the following * construct: * * buf [NoPadLen (buf, len)] = 0x00; */ unsigned int NoPadLen (unsigned char *ibuf, unsigned int ilen) { unsigned int i; // adjusted length unsigned char *p; // pointer to last character p = ibuf + (ilen - 1); i = ilen - (unsigned int) *p; return (i); } // end aesencdec.c =================================================================== #!/usr/bin/perl # aesenc.pl - encrypt data and hex encode using AES CBC with a 256 bit key # # Dave Stoddard - 12/11/2008 # dgs at accelix.com # # This software is placed the public domain and free for any use. # No warrantees. Use at your own risk. # # Description # ----------- # This is a demonstration program for illustrating interoperability between # C and Perl using AES CBC encryption. The program functions as a console # application that prompts the user for input, encrypts and hex encodes the # input, and prints the encrypted result. The result may be decrypted using # aesdec.c or aesdec.pl. To exit the loop, enter "exit" or "quit" at the # prompt. # use strict; use warnings; # use diagnostics; use Crypt::CBC; my $obj = ""; my $len = ""; my $key = "abcdefghijklmnopqrstuvwxyz012345"; my $iv = "RandomIVRandomIV"; my $meth = "Crypt::OpenSSL::AES"; my $encdata = ""; print "input> "; while (<STDIN>) { chomp; last if ($_ eq "exit"); last if ($_ eq "quit"); # create cipher object $obj = Crypt::CBC->new ( -key => $key, -literal_key => 1, -header => "none", -iv => $iv, -cipher => $meth ); # encrypt input data $encdata = $obj->encrypt_hex ($_); $len = length ($encdata); print "encrypted: $encdata (len = $len)\n"; # input prompt print "\ninput> "; } exit 0; # end aesenc.pl =================================================================== #!/usr/bin/perl # aesdec.pl - decrypt hex encoded strings using AES CBC with a 256 bit key # # Dave Stoddard - 12/11/2008 # dgs at accelix.com # # This software is placed the public domain and free for any use. # No warrantees. Use at your own risk. # # Description # ----------- # This is a demonstration program for illustrating interoperability between # C and Perl using AES CBC encryption. The program functions as a console # application that prompts the user for input, decodes and decrypts the input, # and prints the decoded result. To exit the loop, enter "exit" or "quit" # at the prompt. # # NOTE: The key and IV used to decode the input must be the same as the # encoding program (see aesenc.c or aesenc.pl). # use strict; use warnings; # use diagnostics; use strict; use Crypt::CBC; my $obj = ""; my $len = ""; my $key = "abcdefghijklmnopqrstuvwxyz012345"; my $iv = "RandomIVRandomIV"; my $meth = "Crypt::OpenSSL::AES"; my $decdata = ""; print "encrypted hex input> "; while (<STDIN>) { chomp; last if ($_ eq "exit"); last if ($_ eq "quit"); # create cipher object $obj = Crypt::CBC->new ( -key => $key, -literal_key => 1, -header => "none", -iv => $iv, -cipher => $meth ); # decrypt input data $decdata = $obj->decrypt_hex ($_); $len = length ($decdata); print "decrypt: $decdata (len = $len)\n"; # input prompt print "\nencrypted hex input> "; } exit 0; # end aesdec.pl =================================================================== #!/usr/bin/perl # aesencdec.pl - encrypt and decrypt data using AES CBC with a 256 bit key # # Dave Stoddard - 12/11/2008 # dgs at accelix.com # # This software is placed the public domain and free for any use. # No warrantees. Use at your own risk. # # Description # ----------- # This is a demonstration program for illustrating the encryption and # decryption of input data in Perl using AES CBC encryption that is # interoperable with C language programming using OpenSSL. Refer to # aesencdec.c for an equivalent of this program in C. # # The program functions as a console application that prompts for user input, # encrypts and displays the hex encoded result, then decrypts the result and # displays the original input. To exit the input loop, enter "exit" or # "quit" at the prompt. # use strict; use warnings; # use diagnostics; use Crypt::CBC; my $obj = ""; my $len = ""; my $key = "abcdefghijklmnopqrstuvwxyz012345"; my $iv = "RandomIVRandomIV"; my $meth = "Crypt::OpenSSL::AES"; my $encdata = ""; my $decdata = ""; print "input> "; while (<STDIN>) { chomp; last if ($_ eq "exit"); last if ($_ eq "quit"); # create cipher object $obj = Crypt::CBC->new ( -key => $key, -literal_key => 1, -header => "none", -iv => $iv, -cipher => $meth ); # encrypt data $encdata = $obj->encrypt_hex ($_); $len = length ($encdata); print "coded: $encdata (len = $len)\n"; # decrypt data $decdata = $obj->decrypt_hex ($encdata); $len = length ($decdata); print "normal: $decdata (len = $len)\n"; # check for match if ($decdata eq $_) { print "compare OK.\n"; } else { print "MISMATCH!!\n"; } # input prompt print "\n"; print "input> "; } exit 0; # end aesencdec.pl =================================================================== I hope this helps other developers that are working with OpenSSL encryption in multiple environments. Please feel free to ask me questions and/or send comments. Best Regards, Dave Stoddard Accelix LLC 3316 Kilkenny Street Silver Spring, MD 20904-1735 http://www.accelix.com/ 301-572-2908 : office 301-455-0245 : mobile 301-576-4288 : fax dgs at accelix.com ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org