[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: Partially backported in r87382 (3.1) and r87383 (2.7). Not everything could be merged in because of HTTP 0.9 support and (in 2.7) a slightly different architecture. Thank you. -- stage: patch review -> committed/rejected status: open -> closed __

[issue6791] httplib read status memory usage

Senthil Kumaran added the comment: In the morning, I had a comment on the patch wondering why read _MAXLENGH + 1 and then check for len of header > _MAXLENGH. Instead of just reading _MAXLENGH (and if the length matched rejecting). ( Looks like it did not go through). I think that either way

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: Here is a patch limiting line length everywhere in http.client, + tests (it also affects http.server since the header parsing routine is shared). -- stage: needs patch -> patch review Added file: http://bugs.python.org/file20100/httplinelength.patch _

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: Now that 0.9 client support has been removed, this can proceed (at least for 3.2). -- ___ Python tracker ___ __

[issue6791] httplib read status memory usage

Senthil Kumaran added the comment: On Thu, Dec 16, 2010 at 02:02:10PM +, Antoine Pitrou wrote: > I don't think you understood the issue here. Calling readline() without > a maximum length means the process memory potentially explodes, if the > server sends gigabytes of data without a single

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: > It does. Doesn't it? Because I saw in your patch that you fall back on > HTTP 1.0 behaviour when the server does not return a status line and > in which case a Exception will be raise and this issue won't be > observed. I don't think you understood the issue

[issue6791] httplib read status memory usage

Senthil Kumaran added the comment: On Thu, Dec 16, 2010 at 01:18:30PM +, Antoine Pitrou wrote: > Well, removing 0.9 support doesn't make this obsolete, does it? It does. Doesn't it? Because I saw in your patch that you fall back on HTTP 1.0 behaviour when the server does not return a status

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: Well, removing 0.9 support doesn't make this obsolete, does it? -- status: pending -> open ___ Python tracker ___ __

[issue6791] httplib read status memory usage

Senthil Kumaran added the comment: I just read the whole discussion and it seems that code was in place so that client can tolerant of a BAD HTTP 0.9 Server response. http://www.w3.org/Protocols/HTTP/OldServers.html Given that issue10711 talks about removing HTTP/0.9 support (+1 to that), this

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: > That's true. Near the bottom of the code, it says: > > # The status-line parsing code calls readline(), which normally > # get the HTTP status line. For a 0.9 response, however, this is > # actually the first line of the body! > > Limiting the length of

[issue6791] httplib read status memory usage

Ross Lagerwall added the comment: That's true. Near the bottom of the code, it says: # The status-line parsing code calls readline(), which normally # get the HTTP status line. For a 0.9 response, however, this is # actually the first line of the body! Limiting the length of the status lin

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: By the way, looking at the code, readline() without any parameter is used all over http.client, so fixing only this one use case doesn't really make sense. -- stage: unit test needed -> needs patch ___ Python tracke

[issue6791] httplib read status memory usage

Antoine Pitrou added the comment: First, I don't think the resource module needs to be used here. Second, I don't see why getcode() would return 200. If no valid response was received then some kind of error should certainly be raised, shouldn't it? -- nosy: +pitrou _

[issue6791] httplib read status memory usage

Ross Lagerwall added the comment: A py3k patch against revision 87228. -- Added file: http://bugs.python.org/file20049/i6791_py3k.patch ___ Python tracker ___ ___

[issue6791] httplib read status memory usage

Ross Lagerwall added the comment: Attached is a unit test which tests the issue. Unfortunately, since it uses the resource module to limit memory to a workable size, it will only work on Unix. The given patch appears to fix the issue well. I think this should be taken as a security issue (eve

[issue6791] httplib read status memory usage

Changes by Senthil Kumaran : -- assignee: -> orsenthil nosy: +orsenthil ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue6791] httplib read status memory usage

Mark Lawrence added the comment: Sumar, to get this moved forward could you please provide a unit test. -- nosy: +BreamoreBoy stage: -> unit test needed versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.4, Python 2.5, Python 2.6 ___ Python tr

[issue6791] httplib read status memory usage

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue6791] httplib read status memory usage

sumar added the comment: I've also check patch against code in svn tree: wget http://svn.python.org/projects/python/trunk/Lib/httplib.py patch -p0 -i httplib.patch --dry-run patching file httplib.py Hunk #1 succeeded at 209 (offset 54 lines). Hunk #2 succeeded at 303 (offset 10 lines).

[issue6791] httplib read status memory usage

New submission from sumar : During writing some code I discovered some behaviour of httplib. When we connect to host, which doesn’t respond with status line, but it just sending data, httplib may consume more and more memory, becouce when we execute h = httplib.HTTPConnection(‘host’) h.conect() h