2009/2/18 Ger Hobbelt <g...@hobbelt.com>
> On Tue, Feb 17, 2009 at 12:49 PM, sofian sindhi <sofiansis...@gmail.com>
> wrote:
> > Dear all:
> > I trace openssl recently and I cannot find where the location of
> EVP_CIPHERs
> > defined in evp.h.
> > In evp.h, it is declared as const EVP_CIPHER *EVP_camellia_128_ecb(void);
> > But where is the trully EVP_camellia_128_ecb(void) located?
> > I have grepped the source but I cannot get any hint.
>
> In your question you are mixing up two things: EVP_CIPHER is a data
> structure type (used to carry around cipher context needed by the user
> & the cipher to perform the task in multiple phases while the data is
> fed piecemeal to the system at the same time).
>
> grep EVP_CIPHER -->
> ossl_typ.h:
> typedef struct evp_cipher_st EVP_CIPHER;
> --> grep "struct evp_cipher_st" -->
> evp.h:
> struct evp_cipher_st { ...... };
>
> (which, incidentally, also has /* EVP_CIPHER */ as a comment/hint at
> the end, so we are heartened in knowing we've hit the jackpot there)
>
>
> When, instead, you want to find where the _function_
> EVP_camellia_128_ecb() is defined in the code, and, as you found,
> cannot track it's definition down using grep (regular text search),
> then you can go about it various ways, but the oldest and most sure
> way is this:
>
> when you are in such a situation (and otherwise ;-) ), remember that
> 'C' also comes with a *preprocessor*.
> That one is used to conditionally include / exclude chunks of source
> code at compile time, but also offers 'macro' functionality, which in
> the 'C' realm, can be used to implement something similar to
> 'templates' (as we know them in, for example, 'C++'. FYI: OpenSSL
> uses this possibility to great effect in various spots (EVP & ASN1
> most particularly).
> Nothing fancy or difficult, it's just that people tend to forget or,
> more often, never got taught the language properly (and then complain
> about others' lack of minding their forgetfulness :-) ). No worries,
> it's just got to be my Grumpy Day today. ;-)
>
> So, can't find a function or data definition anywhere?
>
> Here's the general process of digging them up:
> Run the code through the *preprocessor* and then have a grep through
> that (intermediate) output. (After all, all the preprocessor does is
> munch source code and spit out 'preprocessed' *source code* for the
> actual *compiler*.
>
> (Almost) all compilers (and at least ALL the ones I've met in
> non-embedded environments) offer an option to do exactly that:
> generally, this is command line option '-E' (~ 'execute preprocessing
> phase only').
> for example:
> gcc has '-E'
> MSVC cl has '-E' and '-EP'
>
> the hard(er) part is getting the '-E' option added to your compiler
> commandline argument set (easy on most systems by temporarily
> redefining GCC or CC environment args so that
> CC='gcc -E'
> instead of
> CC='gcc'
> but this can cause some trouble in your makefiles as they won't expect
> this, but alas, you're interested in preprocessed sourcecode there
> now, not working executables, so ignore the alarm bells and let it
> rip.
>
> Most systems will write such -E preprocessed output to either a .i
> file (<sourcefilename>.i, e.g. apps.c --> apps.i) or the .o or .obj
> file if such a target was specified explicitly on the commandline
> using '-o'. There's no 'always so' rule to this bit, hence me saying
> it's 'harder', meaning: you must check how it acts on your system and
> environment, whichever it is.
>
> The alternative is running your suspects through 'gcc -E' or your
> local equivalent *by hand*. ( = copy&paste commandlines issued from
> make, add the -E yourself and have a go.)
>
>
> The second alternative, when '-E' is not available or otherwise
> unwilling to cooperate, is to trace the code using a source-viewing
> debugger: as you run through the code you will notice that function
> calls will instead sometimes show macro invocations outside any
> function scope. That's a sure sign there's some templating / code
> generation going on under the hood right there.
> --> from there, find those macro definitions and inspect their source
> code. Further drilldown takes a tool like grep again.
>
>
>
>
> So far, Preprocessor Ed 102. (Giving you just the result is pretty
> useless; the above is needed to enable you to reproduce the result and
> answer subsequent, similar questions on your own.)
>
> The result of this search will find you EVP_camellia_128_ecb() here:
>
> evp_locl.h:
> const EVP_CIPHER *EVP_##cname##_##mode(void) { return &cname##_##mode; }
> which is a function returning the static/const structure defining the
> cipher and its requirements, such as block length, IV size, ASN.1
> aspects such as object ID (nid), plus: the functions (in object/class
> oriented terms that would be 'methods' rather) used to perform the
> action on chunks of data (cname##_##mode##_cipher), cleanup the
> EVP_CIPHER object when you're done (cleanup), etc.
>
> When you follow the preprocessor (i.e. answer the question: "how did
> this bit get expanded into that EVP_camellia_128_ecb() we were looking
> for?"), you'll find it starts here:
>
> e_camellia.c:
> IMPLEMENT_BLOCK_CIPHER(camellia_128, ks, Camellia, EVP_CAMELLIA_KEY,
> .......
>
> which is defined here:
> evp_locl.h:
> #define IMPLEMENT_BLOCK_CIPHER(cname, ksched, cprefix, kstruct, nid, \
> ....
>
> and the part of that macro which leads us further is this bit in there:
>
> BLOCK_CIPHER_defs(cname, kstruct, nid, block_size, key_len, iv_len,
> \
>
> as that expands, thanks to the definition here:
>
> evp_locl.h:
> #define BLOCK_CIPHER_defs(cname, kstruct, \
> ....
>
> via this macro's bit:
> ... \
> BLOCK_CIPHER_def_ecb(cname, kstruct, nid, block_size, key_len, iv_len,
> flags, \
> ......
>
> via
>
> evp_locl.h:
> #define BLOCK_CIPHER_def_ecb(cname, kstruct, nid, block_size, key_len, \
> ...
> -->
> BLOCK_CIPHER_def1(cname, ecb, ecb, ECB, kstruct, nid, block_size, key_len,
> \
> ...
>
> via
>
> #define BLOCK_CIPHER_def1(cname, nmode, mode, MODE, kstruct, nid,
> block_size, \
> ...
> *** BINGO! ***
> ...
> const EVP_CIPHER *EVP_##cname##_##mode(void) { return &cname##_##mode; }
>
>
>
> Granted, first time around, this may give you a headache, unless
> you're like me and wondered, after reading the C language specs
> (before starting to code), where one would use the ## operator to good
> effect in daily practice, and then discover to your extreme joy a
> piece of code where this is used to create 'code templates' in C, very
> much like what we have today as templates in C++. The code which got
> me that happy[*] is long gone (something written close to 20 years ago
> by a man named Sim Yskes for Shell), but here is another prime
> example.
>
> [*] finally, I _learned_ something, instead of hearing the IT
> equivalent again of chewing gum fixing gas tank leaks as they already
> did way back in WWII. Apparently, software in peace time is not
> expected to live any longer than Spitfires in war time, but I digress.
>
>
>
> For your edification:
>
> Another very nice use of this ability to make code manageable yet
> concise (and, yes, we expect one has received (and picked up) an
> education other than primary school alone when going to work on this
> sort of thing) is the typechecking safestack template code (see
> <safestack.h> which wraps compile typechecking around the generic core
> functionality in <stack.h>; C++ coders would, today, do exactly the
> same using templates. So would Java coders or anybody else who's got a
> (strictly typed) language with built-in template ability and a
> need/want for strict type checking.
> This <safestack> is a very simple use of macros this way, so this bit
> of code is preferable to read when you're learning this kind of thing
> for the first time.
>
> When you 'grok' (understand) how the macros in <safestack.h> provide
> this compile-time typechecking ability, you can take it up to the next
> level and either 'read' the EVP macros in evp_locl.h and see how they
> are used, or better (because the format is a general standard so you
> can read up anywhere about what it should do, so you can think about
> how you'd do it and see how the OpenSSL creators have solved this
> problem) read the ASN.1 code, which is another example of typesafe
> template code [generation] in C, just like EVP.
> (for ASN.1 study, start at the ans1parse.c code, from there it'll take
> you to tasn_typ.c and asn1t.h, among others.)
> (for EVP study, start at apps/enc.c, which will lead you to evp_enc.c
> and consequently, evp.h and evp_locl.h.)
>
>
>
> TIP: the MSVC IDE has a nice feature, which works once you use project
> files for this stuff (available at hebbut.net, for example): click on
> a function or type, right-click -> 'Go to definition' and it jumps to
> the line and sourcefile where it believes that bit has been defined.
> (Same for declarations, by the way.) It's not a 'sure thing' as the
> IDE can be easily misled, but generally, when it lands on such a macro
> instead of the name proper, it takes only a little thought and further
> drilldown using right-click->"go to definition", to find where the
> structure or function name is constructed from its parts, using ##.
> That is, at least, how I did it while writing this. ;-) (grep works
> too, but I'm lazy and this is just a tad faster)
>
>
>
>
>
> I wrote this in hopes I only need to write it once, as others arriving
> at that same junction where you are: "where the heck is that function
> gone now?!" will read this and know what to do next after all. Some
> stuff requires a longer answer to really get you something. (And next
> time, I can refer back. Saves me the trouble of typing tomorrow.)
> The key to it all is standard C 'preprocessor parameter concatenation'
> a.k.a. ## (plus students reading language (grammar) specifications
> instead of having to stumble forward by trial & error as they receive
> 'practice-oriented' ed).
>
> Dear Ger:
Thanks for your PLENTY help. :)
I have to digest first.
Sincerely Yours,
sofian sindhi
