[issue1508475] transparent gzip compression in urllib

Changes by Demian Brecht : -- nosy: +demian.brecht ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue1508475] transparent gzip compression in urllib

Martin Panter added the comment: I suggest resolving Issue 15955 first, then the GzipFile API could be used without fear of decompression bombs. -- ___ Python tracker ___

[issue1508475] transparent gzip compression in urllib

Changes by Berker Peksag : -- nosy: +berker.peksag ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue1508475] transparent gzip compression in urllib

Martin Panter added the comment: The Lib/xmlrpc/client.py file appears to already support compression using “Content-Encoding: gzip”. Perhaps it could be leveraged for any work on this issue. -- ___ Python tracker

[issue1508475] transparent gzip compression in urllib

Martin Panter added the comment: Related: Issue 1243678, which includes a patch for “httplib” (now known as “http.client”?). That patch looks like it sets Accept-Encoding and decodes according to Content-Encoding. However I suspect it is also trying to be too “transparent” at the wrong level a

[issue1508475] transparent gzip compression in urllib

Martin Panter added the comment: I think the patch is indeed a bit short, for instannce it looks like calling read() without a size limit could bypass the decoding. Also, I wonder if Content-Encoding handling is better done at a higher level. What if someone wants to download a *.tar.gz file?

[issue1508475] transparent gzip compression in urllib

Changes by Berker Peksag : -- versions: +Python 3.5 -Python 3.3 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe

[issue1508475] transparent gzip compression in urllib

Jim Jewett added the comment: This is an enhancement, so I am changing the affected version from 3.3 to 3.5. It is python-only, which works well with the cheeseshop. That said, the patch is truly short; if that is really sufficient, it could almost go into the documentation as a recipe. But

[issue1508475] transparent gzip compression in urllib

Raymond Hettinger added the comment: Victor, the patch looks good and would be a welcome enhancement. There should be an option for turning this on and off (perhaps, I want the zipped content and want to unzip later or in a different thread). Consider adding support for "deflate" as well. ---

[issue1508475] transparent gzip compression in urllib

Martin Panter added the comment: I have code that already handles an “gzip” encoded response from urlopen(). All three patches leave the Content-Encoding header intact, so I suspect my code would try to decompress the body a second time. Deleting this header (as already suggested) would work f

[issue1508475] transparent gzip compression in urllib

STINNER Victor added the comment: I updated issue1508475.diff for Python 3.4 and removed the change in urllib: http_client_gzip.patch. The patch only changes http.client to support server sending gzip data. For example, the new python.org website serves gzip data even if the Accept-Encoding h

[issue1508475] transparent gzip compression in urllib

Changes by John O'Connor : -- nosy: +jcon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue1508475] transparent gzip compression in urllib

Changes by Stephen Thorne : -- nosy: +jerub ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue1508475] transparent gzip compression in urllib

Senthil Kumaran added the comment: I think, the transparent compression should work at http.client level. I also agree with other points made by Serhiy: - transparent decompression should delete headers Content-Encoding and Content-Length (this is as per RFC too) - Should not do another comp

[issue1508475] transparent gzip compression in urllib

Serhiy Storchaka added the comment: The first step is to answer on the fundamental question: on what level transparent decompression will work? On http.client level or on urllib level? Patch for first case will be much more difficult, but will benefit from compression in other http-based prot

[issue1508475] transparent gzip compression in urllib

Changes by Raymond Hettinger : -- priority: normal -> high ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue1508475] transparent gzip compression in urllib

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue1508475] transparent gzip compression in urllib

Éric Araujo added the comment: Enabled by default with a knob to turn it off sounds good. Maybe the original headers could be preserved in some object. -- keywords: -easy, patch ___ Python tracker ___

[issue1508475] transparent gzip compression in urllib

Tom Pinckney added the comment: What if this gzip decompression was optional and controlled via a flag or handler instead of making it automagic? It's not entirely trivial to implement so it is nice to have the option of this happening automatically if one wishes. Then, the caller would be a

[issue1508475] transparent gzip compression in urllib

Serhiy Storchaka added the comment: The patch for py3k also has the disadvantage that the content is decoded even if the user has defined a Content-Encoding and he is going to process compressed response himself. -- ___ Python tracker

[issue1508475] transparent gzip compression in urllib

Senthil Kumaran added the comment: In that case, transparent decompression should not be available. ( Request header should not be sent and response wont be compressed). -- ___ Python tracker ___

[issue1508475] transparent gzip compression in urllib

Serhiy Storchaka added the comment: What if the gzip module is not available? I think, with transparent decompression should delete headers Content-Encoding (to free the user from re-decompression) and Content-Length (which is wrong). -- ___ Python

[issue1508475] transparent gzip compression in urllib

Changes by Serhiy Storchaka : -- nosy: +storchaka versions: +Python 3.3 -Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mail

[issue1508475] transparent gzip compression in urllib

Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue1508475] transparent gzip compression in urllib

Senthil Kumaran added the comment: Patch for py3k. -- stage: unit test needed -> patch review Added file: http://bugs.python.org/file19811/issue1508475.diff ___ Python tracker ___

[issue1508475] transparent gzip compression in urllib

Senthil Kumaran added the comment: The transparent gzip Content-Encoding support should be done at the http.client level code. Before adding this feature, a question needs to be sorted out. If we support the transparent gzip and wrap the file pointer to a GzipFile filepointer, should reset the

[issue1508475] transparent gzip compression in urllib

Changes by Nadeem Vawda : -- nosy: +nvawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue1508475] transparent gzip compression in urllib

Changes by MunSic JEONG : -- nosy: +ruseel ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue1508475] transparent gzip compression in urllib

Changes by Éric Araujo : -- components: -Extension Modules nosy: +eric.araujo -BreamoreBoy title: transparent gzip compression in liburl2 -> transparent gzip compression in urllib ___ Python tracker ___