New submission from Patrick Michaud: Using python's httplib, there is no way to set a timeout for the full request cycle. A timeout can be given, which will apply to the connection attempt and each blocking socket operation. However, a malicious (or poorly performing) server can keep a connection open for an indefinite amount of time by slowly sending data. Consider this server process:
https://gist.github.com/vegitron/bc883ddc88fe9253adc3e0bccea6445e and this client: https://gist.github.com/vegitron/4ee269b6492ff80d350e108363689d5c With a timeout of 0.5, the client takes 0.501363039017 seconds. With a timeout of 2.5, it takes 10.0041370392 seconds. This is explained in the documentation, but it's a problem. A commonly suggested solution is to use SIGALRM to set a timeout, but that doesn't work in a multi-threaded environment. Moving to multi-process introduces big slow downs as I can't use connection pools, and data needs to be serialized and deserialized for the parent process. I would like to propose an addition to httplib that would add a hook to httplib.HTTPResponse._read_chunked (python 2) and http.client.HTTPResponse._readall_chunked (python 3) that would either: 1) support an overall read timeout, by tracking a per-response start time and raising a timeout exception if that chunk read finishes after the given timeout 2) support a per-chunk callback function, so a client of httplib/http.client could define that logic for themselves. Current possible timeouts, and where they can happen: connect | read chunk | read chunk | read chunk [ timeout ] [ timeout ] [ timeout ] [ timeout ] Proposed addition: connect | read chunk | read chunk | read chunk [ timeout ] [ timeout ] [ timeout ] [ timeout ] [ total read time out ] ---------- messages: 286807 nosy: Patrick Michaud priority: normal severity: normal status: open title: add socket independent timeout to httplib/http.client read type: enhancement versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29422> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
