I have created an encryption layer for one of our products that uses RC4 from OpenSSL's libcrypto.a. I have found, however, that when I pass these encrypted outputs from a UNIX platform to an x86 platform, I am unable to decrypt properly past a certain bit strength. I am using OpenSSL-0.9.6j, but I do not believe the version will make difference.
For example, when I pass RC4 encrypted data from Linux to Win32, it works all the way up to 2048-bit RC4 encryption.
However, from Linux to Solaris 8, or Win32 to Solaris 8, I can pass up to 144-bit encrypted data and things will work. However, anything past 144-bit strength encryption will not work.
Linux to Linux, Win32 to Win32, and Solaris to Solaris all work.
I have discovered the RC4_INT define and on the UNIX platforms this is defined to be unsigned char whereas on the x86 platforms it is unsigned int.
if I change that from unsigned int to unsigned char, then things will work, but in opensslconf.h, it specifically warns me not to do that for the P6 platforms.
So my questions are:
1) Is there an issue with passing RC4 encrypted data > 144-bit between platforms? If not, why does encryption upto 144 bits/18 bytes work but nothing greater? Does it have something to do with byte-ordering, int-sizing, realigning boundaries, etc?
2) How can I get the Configure perl script to recognize the RC4_CHAR parameter? When I try to build on Linux (Redhat Advanced Server 2.1) straight from the just-tarred openssl-0.9.6j directory I get the following:
I find that I have to manually edit the Configure file so that RC4_CHAR is added and the opensslconf.h file is modified. Is it legit to hand-modify the Configure script to set RC4_CHAR? I want to ensure that I have a completely legitimate and reproducible form of the openssl library, and I would prefer that I can use only the vanilla Configure file to generate our libcrypto.abash-2.05$ ./Configure RC4_CHAR linux-elf target already defined - RC4_CHAR bash-2.05$
3) Is it safe to define RC4_INT unsigned char on all platforms? I'm not sure if the P6 reference in opensslconf.h means strictly Pentium II class machines (which I remember had a problem running 16-bit applications), or anything above Pentium-class machines.
Thanks for any help!
Terry
_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail
______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
