David Schwartz wrote:
Jason Pettiss wrote:
I have a server which reads/writes a socket independently; that is to
say, at the same time (not a request-response model). I note in the
FAQ it says I must not allow multiple threads to use an SSL connection,
so clearly if my sockets are blocking I cannot support full-duplex
traffic (because I cannot call SSL_write while an SSL_read is blocking,
for instance).
It's important that I be able to read a packet as soon as one is
available, and at the same time, send a packet as soon as I have one to
send... I would not want to delay the send until a pending read were
complete for example.
I'm uncertain whether placing the socket into non-blocking mode will
actually help here: if an SSL_read returns telling me I need to call it
again later, is it alright to go ahead and start a new SSL_write
operation?
That's not what SSL_read will tell you. SSL_read will tell you that it
cannot make further forward progress until something happens. You can call
SSL_read at any later time you wish. The report that it cannot make forward
progress is just a hint.
The only quirks are with SSL_write. You must set
SSL_ACCEPT_MOVING_WRITE_BUFFER (unless you are sure your write buffer will
never move). And you must present a consistent data stream to SSL_write. (So
you can't try to send 'FOO', get 1 back, and later try to send anything that
doesn't start with 'OO'.)
But this flag (while documented to the contrary) does nothing inside
libssl. So yes the documentation says you should set it, prove to me
that OpenSSL behaves in a different way because you set it.
A hint to DS: grep the source tree of OpenSSL and follow all the
code-paths determined by this flag to their conclusion.
Also I'm wondering if the limitation of not being able to write/read at
the same time in blocking mode is easily overcome, for example by
preventing re-negotiation (my application is on both ends of the pipe
here), or by replacing the read/write BIOs, or by supplying some
magical mutex callback function or something.
Blocking mode is way more trouble than it's worth. I would just ditch it,
and all the problems it causes, once and for all. Then never look back.
My own thoughts on the Original Posters comments are:
* The OpenSSL API does indicate the threading issues with your
proposed usage. It is true to say that if you serialize the usage of
any 'SSL *' instance with respect to itself then you will never
experience a usage/threading problem. This is to say that two (or more)
threads can each independently operate the OpenSSL API with _DIFFERENT_
'SSL *' instances at the same time (without regard for one another).
* Now the next question you might want to ask, "is it allowed for
exactly two threads to operate specifically the SSL_read() and
SSL_write() on the _SAME_ 'SSL *' instance at the same time ?" My
understanding would be that the answer is NO. This is a limitation in
the OpenSSL library, since some of the shared parts of 'SSL *' have no
protection and the SSL_read() and SSL_write() code-paths have not been
audited/reworked to minimize the contention/data-race issues.
However this does not exclude the use of OpenSSL for full-duplex operations.
You need to separate your 3 concerns:
* The desire to process incoming data as soon as possible.
* The desire to send outgoing data as soon as possible.
* The desire to have your application go to sleep when neither of the
above is possible and the desire for your operating system to wake up
your application as soon as some condition changes which _MIGHT_ make it
possible for one of the first 2 points (read/write) to take place now.
The 'read' case)
Well this is already covered in both blocking and non-blocking usage,
your application gets back control (to process data) as soon as data can
be processed.
The 'write' case)
Well this is already covered in both blocking and non-blocking usage,
your application gets back control (to create more data to send) as soon
as the layer below OpenSSL (usually the OS kernel buffering) has stored.
The 'sleep/wakeup' mechanism)
Well this is clearly an issue of blocking verses non-blocking. There
is a clear case that you _MUST_ use blocking IO here (this is despite Mr
Schwartz's comments otherwise). The reason you must use non-blocking
is that in order to satisfy concerns 1 and 2 you can-not possibly let
the operating system block your application from having control of the
'SSL *' because (if you remember from the comment 2 I made right at the
start) the OpenSSL API does not let you operate SSL_read() and
SSL_write() on the _SAME_ 'SSL *' instance at the same time. So if some
other thread is stuck and asleep in the middle of using 'SSL *' then
it is unsafe for you to use it from another (unblocked) thread.
So to me there is no clear way to use blocking IO once all the facts
are considered with your intended usage and your design criteria.
The only other comment I can make is that both the SSL_read() and
SSL_write() calls have a soft-error return for when no further work
(progress) can be made. It is at this point you perform your 'sleep'
function and indicate to the OS which events you want that sleep to be
woken by.
This is based on your applications intent, usually an application is
always ready to read in more data (but internal buffering and memory
exhaustion considerations should be made), so it usually indicates to
the OS to wake me up if more data is available to read.
Your application then also has to evaluate its intent to send data, you
don't always have something more to send. If you do then you need to
indicate to the OS to wake me up if I can push more data down into the
kernel buffer.
You then call your OS sleep function with the appropriate wakeup events
(and possible maximum timeout).
You can then keep looping around this basic IO sleep/wake cycle.
Darryl
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
