On Aug 15, 2013, at 10:38 PM, Nico Williams wrote:
Hmm, I've only read the article linked from there: http://android-developers.blogspot.com/2013/08/some-securerandom-thoughts.html
Yeah, that's the only place I've seen it, and then the Google+ thread I linked to is essentially the comment area for that post. We (meaning those of us commenting in the thread) haven't gotten any official answer from Google, but Nikolay Elenkov has been very helpful in reconstructing what seems to be happening. We've exchanged a few more posts this evening, and it appears that what's happening is that OpenSSL is correctly self-seeding when system_server starts, but then system_server forks (without execing) to start multiple processes, and these processes are producing the same random sequence. It's not yet entirely clear why, since the OpenSSL source code looks like it's trying to be fork-safe, but it appears that somehow in practice it's not succeeding.
I'll take a look. For now it seems there should be no need to set any thread-related callbacks, no? Or if they are needed, we should make them no-ops on OSes with thread libraries.
Threading callbacks are currently needed for any application that isn't single threaded. (The situation is still the same as what's covered in the 2002 O'Reilly book, "Network Security with OpenSSL." It has some sample code for setting them up on Windows and pthreads.) I totally agree with you that I'd love to see support for at least pthreads and Windows threads baked into OpenSSL, much like libevent does, for example. But I suspect that convincing the OpenSSL developers will be an uphill battle.
(Just to be clear about my role: I'm an OpenSSL user, an occasional contributor to the OpenSSL wiki, and also one of several admins on the wiki, simply because I was interested and volunteered. But I'm not an OpenSSL developer, nor do I have a direct line to any of the developers. Nor do I speak for anyone but myself. So this is just me throwing opinions and speculation from the peanut gallery.)
I think that a crypto library should have no worse initialization/finalization/thread safety/fork safety semantics than libpkcs11 in Solaris/Illumos, which: a) thread-safely ref-counts C_Initialize()/C_Finalize() calls, b) leaves locking around objects to the app, c) re-initializes (and loses all objects) on the child-side of fork(), d) no thread/lock callback setters. (It's necessary to finalize all objects that refer to sessions for crypto coprocessors, TPMs, tokens, as well as any other stateful objects on the child side of fork(), either that or establish new sessions. It shouldn't be necessary to finalize key objects, say, but hey, it's what PKCS#11 requires.) Better yet: implied initialization (using pthread_once()).
I'm in total agreement with you on that, although even libraries that I consider pretty good in that regard, like libevent and GnuTLS, don't quite achieve that.
A lot of people seem to love static linking. There are problems with dynamic linking, to be sure, but static linking with dynamic dlopen() is insane, and anyways, static linking unfortunately means having to keep track of total dependencies at the program link-edit stage, which is also basically insane (though at least that could be fixed).
Agree. And yeah, I hadn't been thinking about static libc. OS X, at least, has the (mostly) nice property that libc (called libSystem) can't be static.
Oh. Is there any reason not to blow that away, or at least build-time select which to use?
I'm in agreement with you; I just don't think you're going to get the OpenSSL folks on board. They'll probably say something like "we want to be totally agnostic to threading library" without acknowledging that pthreads and Windows threads cover the vast majority of modern mainstream operating systems.
Great. I was hoping that the response wouldn't be something like "no way, we need these callback setting functions for XYZ reasons" or, worse, "no way".
Unfortunately, I think the response will be that. (The OpenSSL folks just haven't weighed in on this thread yet.) That's why I was floating the idea of writing an unofficial companion library that would smooth over these rough spots and provide a "batteries included" approach to people who want it, without having to convince the OpenSSL project to change the core library, which I think would be an uphill battle at best.
--Patrick ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
