Thanks for the comments. I have no problem using aysnc api's, but I inherently
have a problem polling, which, if I read your comments correctly, you are
suggesting in either case (read->write->'sleep a bit'->repeat, or select 'with
a short timeout'->repeat). If I have LOTS of connections (1000+) with light
traffic, then that's LOTS of polling going on to be responsive on any one
connection. I should do the math on the poll overhead, but it sounds bad on
the surface.
So, assuming the OpenSSL stack isn't sneaky and needing to do stuff without the
behind the application's back, what do you think about my option 5 below...
keeping in mind I'm targeting a lot of connections?
That is:
thread_for_each_connection ->
while (true) {
block on condition
grab mutex
check for read
check for write
release mutex
}
thread_for_select_on_all ->
while (true) {
block on select(read of >>all<< connections)
for(i=0;i<numconnections)
if read
signal connection
}
thread_with_something_to_write ->
grab mutex
add to write buffer
release mutex
signal
Do you see any gotcha's with this type of approach?
Mark.
--- Andrew Mann <[EMAIL PROTECTED]> wrote:
> You'd most likely want to use Asynchronous IO. Check for data to read,
> check for data to write, sleep a bit, repeat.
> OpenSSL doesn't (unless it's really sneaky and does it when I'm not
> looking) kick off any other threads. I'm pretty sure it doesn't install
> its own signal handlers either. So there is no way for OpenSSL to do
> anything without your code (or something called from your code) calling
> into OpenSSL. select() surely doesn't. I don't believe TCP sends
> heartbeats either, that's an application thing (or at least higher layer
> than TCP - which usually means some piece of the application).
> If there's some reason you don't want to use Asychronous calls, you
> could look at pselect(), or try blocking on select() with a short
> timeout. I'm not entirely sure if the ssl read calls have a minimum
> amount of data they need to receive before they return. If they do,
> then an indication from select() that data is ready to read doesn't mean
> that a blocking ssl read call won't block waiting for more data.
> The reason usually given to avoid Async IO is complexity, but this type
> of situation is just what it's good for.
>
>
> Andrew
>
>
> Mark Pietras wrote:
>
> > Following up on an older thread, it seems that simply mutex protecting the
> > read/write operations on different threads has solved my problem.
> >
> > Not being one to leave well-enough alone though, I've gone back recently to
> > look at this again and I'm left wondering... how would one do this on a
> single
> > thread??
> >
> > For example, in a single thread connection (ssl reads and writes all on the
> > same thread for a single connection) but multithreaded application, it's
> easy
> > to conceptualize. The connection thread blocks on read. If it has data,
> get
> > it and process it. If some other thread needs to write on that connection,
> > grab a lock on a buffer, buffer the output data, then signal the connection
> > thread that it's got something to do (i.e. write). But in practice, I
> don't
> > see how it will work. My options as I seem them:
> >
> > 1) block on BIO_read. Well, this doesn't work because when the application
> has
> > something to write, there's no way to unblock the read. (or is there?)
> >
> > 2) block on select(read/write). This doesn't work because on an idle
> > connection, "write" will pop us out of select and we'll spin and chew cpu.
> >
> > 3) block on select(read). This doesn't work because if I need to write
> > something from an application perspective, I can't unblock the select
> (unless I
> > put some other FD in the select to trigger it out, but that's messy because
> now
> > I'd then need two per connection-- one for the socket and one for the
> signal;
> > or is there a way to unblock the select with the socket FD?). Not to
> mention,
> > as I understand it, if the openssl stack has something it needs to write,
> it
> > doesn't have the opportunity because I'm blocked at the application level
> on
> > the select. Is this correct? Are there NO circumstances where the openssl
> > stack "decides" it needs to write something on a connection "out of the
> blue"?
> > I know TCP does heartbeats, but I don't know about ssl/tls and what's going
> on
> > in that layer.
> >
> > 4) have the ssl thread block on a condition. have another thread that does
> a
> > select(read). if the select thread finds something to read or any other
> thread
> > has something to write, just signal the condition. then the ssl thread
> tries
> > both reads and writes until it has nothing left to do, then blocks on the
> > condition again. Lot of work for a single connection, but could be
> efficient
> > for multiple connections. This seems to work okay as long as my assumption
> > above about openssl not "deciding" it needs to write is correct.
> >
> > 5) other options/suggestions?
> >
> > Thanks again for your help... Mark.
> >
> >
> >
> >
> > --- "Dr. Stephen Henson" <[EMAIL PROTECTED]> wrote:
> >
> >>On Wed, Aug 27, 2003, Mark Pietras wrote:
> >>
> >>
> >>[code snipped]
> >>
> >>>My connection drops with a "SSL read error: 0" sent to the log. Again,
> >>
> >>it’s a
> >>
> >>>multi-threaded single-process project. The project is portable between
> >>
> >>Windows
> >>
> >>>and Linux; I have the same problem on both platforms. Each sslio BIO
> >>>connection has a reader thread (main loop above) and a shared writer
> thread
> >>>(not shown... basically writes when there’s data to write and checks
> >>>BIO_should_retry() to see if there’s a backlog, which I’ve never seen).
> As
> >>
> >>a
> >>
> >>>side note... in the FAQ "Is OpenSSL thread-safe?", the answer is "Yes
> (with
> >>>limitations: an SSL connection may not concurrently be used by multiple
> >>>threads)". Does this mean a reader and seperate writer thead cannot
> >>
> >>operate on
> >>
> >>>the same connection? Could this be my problem?? And if so, can i simply
> >>>protect the read and write operations with a semaphore or is there
> >>
> >>something
> >>
> >>>deeper that requires them to actually be on the same thread???
> >>>
> >>
> >>Running separate reader and writer threads on (presumably) the same BIO is
> a
> >>definite problem which can cause race conditions. For example both threads
> >>receiving a retry condition and one performing an action to reset it before
> >>the other has checked the condition, or both attempting to resolve the same
> >>condition.
> >>
> >>Also a read on a BIO can cause a write to the underlying transport and
> >>vice-versa so you should check that a retry read or write is needed even if
> >>you are only peforming one operation on the SSL BIO.
> >>
> >>Some people have reported success in the past by using a semaphore but I've
> >>not checked the code in any detail to see if this could cause problems.
> >>
> >>Steve.
> >>--
> >>Dr Stephen N. Henson.
> >>Core developer of the OpenSSL project: http://www.openssl.org/
> >>Freelance consultant see: http://www.drh-consultancy.demon.co.uk/
> >>Email: [EMAIL PROTECTED], PGP key: via homepage.
> >>______________________________________________________________________
> >>OpenSSL Project http://www.openssl.org
> >>User Support Mailing List [EMAIL PROTECTED]
> >>Automated List Manager [EMAIL PROTECTED]
> >
> >
> > ______________________________________________________________________
> > OpenSSL Project http://www.openssl.org
> > User Support Mailing List [EMAIL PROTECTED]
> > Automated List Manager [EMAIL PROTECTED]
>
> ______________________________________________________________________
> OpenSSL Project http://www.openssl.org
> User Support Mailing List [EMAIL PROTECTED]
> Automated List Manager [EMAIL PROTECTED]
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
