[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Changes by Etienne Robillard : -- nosy: +erob ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: On 02/01/11 10:50 PM, Glenn Linderman wrote: > Glenn Linderman added the comment: > > Rereading the doc link I pointed at, I guess detach() is part of the new API > since 3.1, so doesn't need to be checked for in 3.1+ code... but instead, may > need to be coded as: > > try: > sys.stdin = sys.stdin.detach() > except UnsupportedOperation: > pass > > -- > > ___ > Python tracker > <http://bugs.python.org/issue4953> > ___ > Hi! using "detach" would be great but I'm missing that method here in 2.7! :-) e...@localhost:~$ python2.7 Python 2.7.1 (r271:86832, Jan 2 2011, 10:38:30) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> sys.stdin.detach Traceback (most recent call last): File "", line 1, in NameError: name 'sys' is not defined >>> import sys >>> sys.stdin.detach Traceback (most recent call last): File "", line 1, in AttributeError: 'file' object has no attribute 'detach' -- title: cgi module cannot handle POST with multipart/form-data in 3.0 -> cgi module cannot handle POST with multipart/form-data in 3.0 ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: i'm thinking this issue is also well connected to: http://bugs.python.org/issue1573931 so a backport of whatever solution comes to 3.2 would be a great addition to Python 2.6 as the very minimum, in order to satisfy minimal backward compatibility! Thanks, On 02/01/11 10:50 PM, Glenn Linderman wrote: > Glenn Linderman added the comment: > > Rereading the doc link I pointed at, I guess detach() is part of the new API > since 3.1, so doesn't need to be checked for in 3.1+ code... but instead, may > need to be coded as: > > try: > sys.stdin = sys.stdin.detach() > except UnsupportedOperation: > pass > > -- > > ___ > Python tracker > <http://bugs.python.org/issue4953> > ___ > -- ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: On 03/01/11 09:45 AM, R. David Murray wrote: > R. David Murray added the comment: > > Etienne: since this is about solving a 3.x specific problem, it will not get > backported. Issue 1573931 looks unrelated to me at a quick glance. FYI, you > will find that you *do* have detach in 2.7 if you open a file using the io > subsystem (import io). Of course, that isn't used for the std files in 2.7. > > Glen: the new IO subsystem is a complete C layer on top of only the most > basic of the C runtime stuff. It does handle cross platform issues. Given > that, and given that the input to CGI *should* be bytes, I think letting an > error raise if the stream is text and detatch isn't available is fine, though > we might find we want to catch it to improve the error message with extra > context. > > Pierre: yes, that diff is what I was looking for. I hope to have time to > look it over later today. > > -- > > ___ > Python tracker > <http://bugs.python.org/issue4953> > ___ > Thanks for theses precisions, David. So will cgi.FieldStorage still be usable in 3.x using 2.5 semantics ? implementing the size argument in the FieldStorage class would surely be a good fix for WSGI middlewares. Either ways (using the new io subsystem) or monkey-patching cgi.FieldStorage so it accepts the size argument could probably helps to resolve memory-usage issues with things like file uploads! Regards -- ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: On 05/01/11 09:12 PM, Glenn Linderman wrote: > Glenn Linderman added the comment: > > Pierre said: > In all cases the interpreter must be launched with the -u option. As stated > in the documentation, the effect of this option is to "force the binary layer > of the stdin, stdout and stderr streams (which is available as their buffer > attribute) to be unbuffered. The text I/O layer will still be > line-buffered.". On my PC (Windows XP) this is required to be able to read > all the data stream ; otherwise, only the beginning is read. I tried Glenn's > suggestion with mscvrt, with no effect > > I say: > If you start the interpreter with -u, then my mscvrt has no effect. Without > it, there is an effect. Read on... > > Antoine said: > Could you open a separate bug with a simple piece of code to reproduce > the issue (preferably without launching an HTTP server :))? > > I say: > issue 10841 > > -- > > Thats a quite annoying response. whats the purposes of a "option" switch if it becomes mandatory ? Are you refering to Windows only users ? I would prefer a way to programmatically allow FieldStorage to use HTTP_TRANSFER_ENCODING if available, to select a matching encoding... Thanks -- ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: yes, lets not complexify anymore please... > Because the HTTP protocol is binary, only selected data, either explicitly or > implicitly (by standard definition) should be decoded, using the appropriate > encoding. FieldStorage should be able to (1) read a binary stream (2) do the > appropriate decoding operations (3) return the data as bytes or str as > appropriate. > > Right now, I'm mostly interested in the fact that it doesn't do (1), so it is > hard to know what it does for (2) or (3) because it gets an error first. > > -- > according to rfc2616... "Transfer-codings are analogous to the Content-Transfer-Encoding values of MIME [7], which were designed to enable safe transport of binary data over a 7-bit transport service. However, safe transport has a different focus for an 8bit-clean transfer protocol. In HTTP, the only unsafe characteristic of message-bodies is the difficulty in determining the exact body length (section 7.2.2 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec7.html#sec7.2.2>), or the desire to encrypt data over a shared transport." I may have not fully understood that part. Is "chunked" encoding what's being used in MIME to allow large file uploads and properly handle multipart POST requests? Thanks, -- Added file: http://bugs.python.org/file20288/unnamed ___ Python tracker <http://bugs.python.org/issue4953> ___ On 06/01/11 04:28 AM, Glenn Linderman wrote: Glenn Linderman mailto:v+pyt...@g.nevcal.com";><v+pyt...@g.nevcal.com> added the comment: Etienne, I'm not sure what you are _really_ referring to by HTTP_TRANSFER_ENCODING. There is a TRANSFER_ENCODING defined by HTTP but it is completely orthogonal to character encoding issues. There is a CONTENT_ENCODING defined which is a character encoding, but that is either explicit in the MIME data, or assumed to be either ASCII or UTF-8, in certain form data contexts. yes, lets not complexify anymore please... Because the HTTP protocol is binary, only selected data, either explicitly or implicitly (by standard definition) should be decoded, using the appropriate encoding. FieldStorage should be able to (1) read a binary stream (2) do the appropriate decoding operations (3) return the data as bytes or str as appropriate. Right now, I'm mostly interested in the fact that it doesn't do (1), so it is hard to know what it does for (2) or (3) because it gets an error first. -- according to rfc2616... "Transfer-codings are analogous to the Content-Transfer-Encoding values of MIME [7], which were designed to enable safe transport of binary data over a 7-bit transport service. However, safe transport has a different focus for an 8bit-clean transfer protocol. In HTTP, the only unsafe characteristic of message-bodies is the difficulty in determining the exact body length (section http://www.w3.org/Protocols/rfc2616/rfc2616-sec7.html#sec7.2.2";>7.2.2), or the desire to encrypt data over a shared transport." I may have not fully understood that part. Is "chunked" encoding what's being used in MIME to allow large file uploads and properly handle multipart POST requests? Thanks, -- Etienne Robillard Company: Green Tea Hackers Club Occupation: Software Developer E-mail: mailto:e...@gthcfoundation.org";>e...@gthcfoundation.org Work phone: +1 514-962-7703 Website (Company): https://www.gthc.org/";>https://www.gthc.org/ Website (Blog): https://www.gthc.org/blog/";>https://www.gthc.org/blog/ PGP public key fingerprint:F2A9 32EA 8E7C 460F 1728 A1A7 649C 7F17 A086 DDEC During times of universal deceit, telling the truth becomes a revolutionary act. -- George Orwell ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: On 10/01/11 05:23 AM, Glenn Linderman wrote: > I'm basing this off the use of Firefox Live HTTP headers tool. > > is sendfile() available on Windows ? i thought the Apache server could use that to upload files without having to buffer files in memory.. HTH, -- Etienne Robillard Company: Green Tea Hackers Club Occupation: Software Developer E-mail: e...@gthcfoundation.org Work phone: +1 514-962-7703 Website (Company): https://www.gthc.org/ Website (Blog): https://www.gthc.org/blog/ PGP public key fingerprint:F2A9 32EA 8E7C 460F 1728 A1A7 649C 7F17 A086 DDEC During times of universal deceit, telling the truth becomes a revolutionary act. -- George Orwell -- title: cgi module cannot handle POST with multipart/form-datain 3.0 -> cgi module cannot handle POST with multipart/form-data in3.0 ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Etienne Robillard added the comment: On 11/01/11 07:36 PM, Glenn Linderman wrote: > Is there a newer standard these browsers are following, that permits UTF-8? > Or even requires it? > > Why is Pierre seeing cp-1252, and I'm seeing UTF-8? I'm running Windows 6.1 > (Build 7600), 64-bit, the so-called Windows 7 Professional edition. > > -- > > May be your browser have differents assumptions on what charset is valid for encoding multipart form data... For instance, all modern browsers allow customizing charsets based on the user's locale. Lastly this behavior is well-defined in RFC 2616, as the "Accept-Charset" HTTP header: "The Accept-Charset request-header field can be used to indicate what character sets are acceptable for the response. This field allows clients capable of understanding more comprehensive or special- purpose character sets to signal that capability to a server which is capable of representing documents in those character sets." just my 2 cents while watching a boring hockey game... :-) -- ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Etienne Robillard added the comment: +1 -- title: cgi module cannot handle POST with multipart/form-data in 3.x -> cgi module cannot handle POST with multipart/form-data in 3.x ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Etienne Robillard added the comment: +1 thanks for this input. I agree for the most part. However if the io semantics in python 3 is radically different than on python 2, I could have expected that WSGI scripts would similarly depend on a newer type of file descriptor access using the ``sys`` module. cheers! -- ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
New submission from Etienne Robillard:
Trying to push to a ssl server but python break in httplib.
erob@nguns:~/django-hotsauce$ hg push
https://tkad...@bitbucket.org/tkadm30/django-hotsauce
pushing to https://tkad...@bitbucket.org/tkadm30/django-hotsauce
warning: bitbucket.org certificate with fingerprint
45:ad:ae:1a:cf:0e:73:47:06:07:e0:88:f5:cc:10:e5:fa:1c:f7:99 not verified (check
hostfingerprints or web.cacerts config setting)
** unknown exception encountered, please report by visiting
** http://mercurial.selenic.com/wiki/BugTracker
** Python 2.7.3 (default, Aug 20 2014, 09:34:08) [GCC 4.7.2]
** Mercurial Distributed SCM (version 3.1)
** Extensions loaded: color, gpg, strip, mq, notify, patchbomb
Traceback (most recent call last):
File "/usr/local/bin/hg", line 43, in
mercurial.dispatch.run()
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 28,
in run
sys.exit((dispatch(request(sys.argv[1:])) or 0) & 255)
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line 69,
in dispatch
ret = _runcatch(req)
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line
138, in _runcatch
return _dispatch(req)
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line
820, in _dispatch
cmdpats, cmdoptions)
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line
600, in runcommand
ret = _runcommand(ui, options, cmd, d)
File "/usr/local/lib/python2.7/site-packages/mercurial/extensions.py", line
196, in wrap
return wrapper(origfn, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/hgext/color.py", line 433, in
colorcmd
return orig(ui_, opts, cmd, cmdfunc)
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line
911, in _runcommand
return checkargs()
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line
882, in checkargs
return cmdfunc()
File "/usr/local/lib/python2.7/site-packages/mercurial/dispatch.py", line
817, in
d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
File "/usr/local/lib/python2.7/site-packages/mercurial/util.py", line 550, in
check
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/mercurial/extensions.py", line
151, in wrap
util.checksignature(origfn), *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/mercurial/util.py", line 550, in
check
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/hgext/mq.py", line 3393, in
mqcommand
return orig(ui, repo, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/mercurial/util.py", line 550, in
check
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/mercurial/commands.py", line
4768, in push
other = hg.peer(repo, opts, dest)
File "/usr/local/lib/python2.7/site-packages/mercurial/hg.py", line 129, in
peer
return _peerorrepo(rui, path, create).peer()
File "/usr/local/lib/python2.7/site-packages/mercurial/hg.py", line 106, in
_peerorrepo
obj = _peerlookup(path).instance(ui, path, create)
File "/usr/local/lib/python2.7/site-packages/mercurial/httppeer.py", line
261, in instance
inst._fetchcaps()
File "/usr/local/lib/python2.7/site-packages/mercurial/httppeer.py", line 58,
in _fetchcaps
self.caps = set(self._call('capabilities').split())
File "/usr/local/lib/python2.7/site-packages/mercurial/httppeer.py", line
172, in _call
fp = self._callstream(cmd, **args)
File "/usr/local/lib/python2.7/site-packages/mercurial/httppeer.py", line
119, in _callstream
resp = self.urlopener.open(req)
File "/usr/local/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/usr/local/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/usr/local/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/local/lib/python2.7/site-packages/mercurial/url.py", line 371, in
https_open
return self.do_open(self._makeconnection, req)
File "/usr/local/lib/python2.7/site-packages/mercurial/keepalive.py", line
255, in do_open
r = h.getresponse()
File "/usr/local/lib/python2.7/site-packages/mercurial/keepalive.py", line
577, in safegetresponse
return cls.getresponse(self)
File "/usr/local/lib/python2.7/httplib.py", line 1028, in getresponse
response = self.response_class(*args, **kwds)
File "/usr/local/lib/python2.7/site-packages/mercurial/keepalive.py", line
380, in __init__
httplib.HTTPResponse.__init__(self, sock, debuglevel, method)
File &qu
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: interpreter is Python 2.7.3. 2.7.8 is buggy! -- ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: Thanks. I have forwarded to Mercurial bugtracker the problem. -- ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: after reviewing with the Mercurial support it was found the issue may be caused by improper python install. However i have been able to reproduce the problem with a fresh install of python 2.7.3 only. -- ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: the bug appear to be reproducible with python 2.7.2 as well. I wonder if --enable-shared is relevant to this problem. My configuration options: ./configure --prefix=/usr/local --enable-unicode=ucs4 --with-threads --enable-shared --disable-ipv6 --with-pymalloc openssl version: OpenSSL> version OpenSSL 1.0.1e 11 Feb 2013 perhaps this openssl version breaks specifically python 2.7.x ? -- ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17085] test_socket crashes the whole test suite
Etienne Robillard added the comment: since TIPC is silently included in python 2.7.3 and recents kernel memory leaks were found in the linux TIPC kernel code, could this result in info disclosure in python when compiled with TIPC ? does the kernel module needs to be loaded to exploit the memleaks from within python ? http://www.openwall.com/lists/oss-security/2013/04/14/3 -- nosy: +erob ___ Python tracker <http://bugs.python.org/issue17085> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: problem is fixed with python 2.7.9. it seem ssl.py module has been greatly improved with this release. Thanks! -- resolution: -> fixed ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: hg push worked like a charm... but for some reasons i get another error trying to install things with setuptools: erob@nguns:~/Desktop/pyaml-14.12.10$ sudo python setup.py install --prefix=/usr/local Processing dependencies for pyaml==14.12.10 Searching for PyYAML Reading http://pypi.python.org/simple/PyYAML/ Download error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) -- Some packages may not be found! Couldn't find index page for 'PyYAML' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ Download error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) -- Some packages may not be found! No local packages or download links found for PyYAML Is setuptools still working with this ssl.py module ? -- resolution: fixed -> ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22235] httplib: TypeError with file() object in ssl.py
Etienne Robillard added the comment: sometimes urllib break at different packages, with "SSL: CERTIFICATE_VERIFY_FAILED" message. It seem related to the new ssl.py backport and pip. sudo python setup.py install --prefix=/usr/local Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.10.tar.gz Traceback (most recent call last): File "setup.py", line 31, in distribute_setup.use_setuptools() File "/home/steiner/Desktop/feedcache-1.4.1/distribute_setup.py", line 145, in use_setuptools return _do_download(version, download_base, to_dir, download_delay) File "/home/steiner/Desktop/feedcache-1.4.1/distribute_setup.py", line 124, in _do_download to_dir, download_delay) File "/home/steiner/Desktop/feedcache-1.4.1/distribute_setup.py", line 193, in download_setuptools src = urlopen(url) File "/usr/local/lib/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/usr/local/lib/python2.7/urllib2.py", line 437, in open response = meth(req, response) File "/usr/local/lib/python2.7/urllib2.py", line 550, in http_response 'http', request, response, code, msg, hdrs) File "/usr/local/lib/python2.7/urllib2.py", line 469, in error result = self._call_chain(*args) File "/usr/local/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/local/lib/python2.7/urllib2.py", line 656, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/local/lib/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/usr/local/lib/python2.7/urllib2.py", line 449, in _open '_open', req) File "/usr/local/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/local/lib/python2.7/urllib2.py", line 1240, in https_open context=self._context) File "/usr/local/lib/python2.7/urllib2.py", line 1197, in do_open raise URLError(err) urllib2.URLError: -- ___ Python tracker <http://bugs.python.org/issue22235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
