> From: owner-openssl-us...@openssl.org On Behalf Of Raj
> Sent: Wednesday, 18 August, 2010 06:49
> I have tried one more method to read the data from the
> socket, which was partially successful it is defined as follows
> do
> {
> dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize);
> // Gets the
> data from the server side
> SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the
> data back to
> the SSL
> } while(dwReadDataLen > 0 );
>
That is my simple until-EOF version, see below.
> By using this method I am able to read the content data from
> the server and
> put it back to my browser. But this method is not consistent though,
> Sometimes browse request will not get completed and also it
> takes lot of
> time complete one browse request
That's pretty vague. One off-the-cuff guess:
As I alluded to, this method has the limitation that it will
only (exit and) close when the server does, so if the request
allows keepalive and the server chooses it, you never turn around
and see if the client=browser is trying again. Maybe it is. If so,
*that* request will never go anywhere. Maybe your browser is
timing out that request and retrying on a new (good) connection.
Maybe this depends on your browser/version/config or request(s),
or even the contents of the response page e.g. script or ActiveX.
You could look at the response data (just the headers is enough)
to see if keepalive is enabled, and check whether your loop
actually exits (i.e. you got EOF, which you *probably* won't
*if* the server chose keepalive). Or you could look at both
responses and requests on the local side with (I think) ssldump.
Or you could use a client which tells you (much) more about
the requests it is making; I guess wget might be persuaded.
Remember there is a big difference between a webpage and a
browser=client request and server response. Typical webpages
have CSS, scripts, images, and sometimes frames and objects.
One webpage may be 10 or 50 or 200 requests and responses.
Often a browser won't show you all of the page, and sometimes
even any of it, until all the requests/responses are complete.
If this is the problem, you need to either:
- get the server to do one response per connection (and close).
I know downgrading the request to 1.0 works, and I'm pretty sure
replacing or adding as applicable Connection: close on 1.1 does.
There may also be server-dependent ways.
- recognize the end of the response and close downward
(and upward also, since this connection is now orphan).
If the browser quickly tries a second request it will
get an error, but (much) faster, and more certainly retry
(since server async close is a more 'expected' error).
- recognize the end of the response and turn around
to handle another request (and response etc.).
> -------------------------
> Replies and quires to the previous posting
>
> > For a socket used with openssl directly, I believe OVERLAPPED
> > will be ignored and is of no use. I think you would have to do
> > your own 'physical' level either as your own BIO type or as
> > a BIO_pair looping back to your code (the more usual way).
> > Frankly I don't think you're anywhere near ready for that.
>
> I didn't understand about this, Can you describe this in more
> detail, Sorry
> for that I am new to this technology
>
See below for the detail I have, which is not very much.
> > You should check for error (<=0) and report/handle it. <snip>
> I have checked all the error codes of SSL functions in my
> application, I have posted only some code snippet to avoid junk data
>
Good. Mention that with your posted code, to avoid getting sidetracked.
> >> SSL_accept(Serverssl);
> > This is useless. SSL_accept _creates_ a server-side endpoint;
> > it is not applicable to a client-side endpoint.
>
> I have removed this from my application
>
> > Also, the data read by SSL_read (like POSIX read or C fread)
> > does not get a null terminator byte added, <snip>
> I have outputted the buffer only for indicative purpose. I
> have removed the
> code for outputting the buffer
>
> > That's your problem. SSL_pending only indicates data _already
> > received and buffered_ by OpenSSL but not yet read by the app. <snip>
> } while(SSL_pending(Serverssl));
>
> Instead of using the above condition I have opted for
>
> while(dwReadDataLen > 0 );
>
> By using this I was able to read the content data.
>
See next.
>
> > For a waited/blocking socket, which is the default as you have here,
> > you need to keep reading from the server (and in your case writing
> > back to the client) until you've done all the data in the response.
> > If you require, or the server chooses, HTTP/1.0 style conn-per-txn
> > (also known as connection: close or not-keepalive or not-pipelined,
> > and also not-chunked) you can just loop until you receive "EOF" (0)
> > from SSL_read, caused by the server closing the connection.
>
> "EOF" (0)
>
> I am not sure about EOF(0), is that some thing similar to
> End Of File in
> C++;
>
It's the same idea as C/C++ but actually more like POSIX/Unix.
C stdio (getchar, scanf, etc.) mostly returns -1* and also sets
a flag that can be tested with feof(fp); except fgets returns null
and fread returns a short count (which can be 0 or >0 but <N).
C++ AFAIK generally just sets a flag which can be tested several
ways, including istream<>::operator bool() (I think that's the
right syntax, I haven't used it for a while, it's at least close).
POSIX/Unix read and recv return 0, and can also return short count
>0 and <N without EOF; OpenSSL follows this convention. Thus
countread==0 means EOF and countread>0 means not-EOF, so that
change is exactly "loop until EOF" as I said *for 1.0 style*.
(*Note: technically the C standard only requires a negative int
value for EOF. In practice everybody always uses -1.)
> > If you allow and the server uses HTTP/1.1 keep-alive (or pipelining)
> > and/or chunked data, the situation can get quite a bit more
> > complicated. See RFC 2616.
> >
> > If you use a nonblocking socket (which is supported on Windows
> > as far as I know but is apparently not the same as OVERLAPPED)
> > you can also do your own timeout -- that is, read until EOF
> > or optionally calculated end of the response body, *or* timeout.
> > Since HTTP servers will normally send a complete response within
> > a short time (like at most a few seconds), and if one doesn't
> > a person at a browser usually doesn't want to wait anyway,
> > this can be a good simple compromise.
>
> Could you send me some code snippet using 'bio' in SSL, I
> have seen using
> 'bio' is some sample applications instead of Sockets
>
First, there's no connection at all between using BIOs and
handling 1.1 (keepalive and chunked). BIOs are 'below' OpenSSL
and HTTP is 'above'. BIOs could be useful for something like
OVERLAPPED sockets; they are of no use at all for 1.1.
Second, you* don't use BIOs instead of sockets (for SSL);
you use them in addition. (You can also use BIOs in OpenSSL
for disk files, like keys and certs, but that's unrelated.)
OpenSSL BIO is a generalized structure that does many things,
but there is one BIO type that wraps a socket namely BIO_sock.
If you just give libssl a socket, it actually creates and
internally uses a BIO_sock, your code just doesn't see it.
You can create the BIO_sock yourself, but there's no point.
(* To be clear, I use 'you' in the US sense of 'anybody',
not necessarily 'Raj' in particular. I understand UK usage
is 'one' and 'one's' etc., but that's unnatural to me.)
What you can do that may be useful, but not easy, is substitute
another BIO that isn't BIO_sock but still does the socket I/O.
To expand what I said before, there are two approaches:
- actually write your own BIO type, which does anything you want;
this will duplicate most of BIO_sock but (presumably) with changes
(such as OVERLAPPED, although I doubt that would actually get you
any benefit over default thread scheduling).
- use BIO_pair, which is a builtin type rather like a Unix pipe;
OpenSSL writes one 'end' and your code gets the data at the other
and must actually send it; conversely your code must actually
receive remote data and pass it in for OpenSSL to read. Since
your code cannot predict when to send and receive, you must
effectively do both 'all the time', either by threads that run
'continuously' (but may block) or by polling logic (which can be
pretty complicated) to respond to whatever happens whenever.
This is all from reading the documentation and apps/*
I don't have code that actually does this; my applications
are fine with the much simpler nonblocking-sockets + select.
(Including a proxy for a protocol much simpler than HTTP.)
<snip history>
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
