Patrick J McNerthney added the comment: Martin,
My understanding of the intention of the "Expect: 100-continue" header would address being able to receive an error response that is determined solely from the http request headers sent. So I do think that that would be the proper way to address that other issue, and without having to worry about sending and receiving data at once. However, what I am running into is a different issue caused by the same root cause of not properly handling "Expect: 100-continue" header in the client. In my case, what is occurring is the URL being called is configured to require an SSL client certificate. Apache is only able to determine that the client certificate is required after the http request headers are sent. Once Apache has the headers and determines that the client certificate is required, the ssl connection is renegotiated. While performing this renegotiation, Apache holds the request data in progress in a "SSLRenegBuffer". If the data sent is larger then the SSLRenegBufferSize configuration, then Apache will fail the request as being to large. With the current HttpsConnection, the sequence of events leading to the failure are: 1. HttpsConnection sends all the headers, including the Expect: 100-continue header, and sends all the 128K of data as fast as the socket allows. 2. Apache receives the request headers and looks at the request URL and determines that a client certificate is required, precipitating a ssl renegotiation. 3. Apache attempts to buffer the data received so far so that the ssl renegotiation can occur. However, the body of the request has already been sent and it is larger then SSLRenegBufferSize. 4. Apache fails the request as being to large. 5. HttpsConnection gets the error response about too large of a request. I have created my own fork of httplib.py that does proper Expect: 100-continue handling, requesting in the following successful steps: 1. HttpsConnection sends all headers, including the Expect: 100-continue header. The Expect: 100-continue header was detected, so HttpsConnection does not send the body yet. 2. HttpsConnection waits for a response. 3. Apache httpd receives the headers and detects that the request url requires a client certificate. 4. The ssl connection is renegotiated using the client certificate. 5. Apache sends the response status of 100. This is the only header sent, in addition to a blank line. 6. HttpsConnection receives the response status 100, which means all is good to go. 7. HttpsConnection sends the body of the request. 8. Apache gets the request body, which now can be very, very large, because it does not have to be held in the SSLRenegBuffer. 9. Apache processes request and returns the response. 10. HttpsConnection receives the response and processes. Note that if Apache detected some error detected with the headers, then the sequence would go like so: 1. HttpsConnection sends all headers, including the Expect: 100-continue header. The Expect: 100-continue header was detected, so HttpsConnection does not send the body yet. 2. HttpsConnection waits for a response. 3. Apache httpd receives the headers and detects some kind of error. Maybe the URL is not handled at all. 4. Apache sends the error response. 5. HttpsConnection receives the error response and uses that as the final response of the request. The body of the request is never sent at all. One unexpected side effect of using the Expect: 100-continue header I found was that requests smaller then the SSLRenegBufferSize were handled by Apache almost 3 times faster then without the Expect: 100-continue header. I assume this is because Apache does have to mess around with all the extra overhead of staging the request in the SSLRenegBuffer, but can instead route the body of the request directly to the request handler. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue1346874> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
