Re: [Python-Dev] cpython: threading.RLock._acquire_restore() now raises a TypeError instead of a
Hi Serhiy, On Fri, Jan 3, 2014 at 8:59 AM, Serhiy Storchaka wrote: >> +if (!PyArg_ParseTuple(args, "(kl):_acquire_restore", &count, &owner)) > > Please don't use "(...)" in PyArg_ParseTuple, it is dangerous (see issue6083 I think that in this case it is fine, because the "k" and "l" are returning C integers. The refcounting issue occurs only when PyObject* are returned. A bientôt, Armin. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Subclasses vs. special methods
Should implicit converting an instance of int, float, complex, str,
bytes, etc subclasses to call appropriate special method __int__ (or
__index__), __float__, __complex__, __str__, __bytes__, etc? Currently
explicit converting calls these methods, but implicit converting doesn't.
class I(int):
... def __int__(self): return 42
... def __index__(self): return 43
...
class F(float):
... def __float__(self): return 42.0
...
class S(str):
... def __str__(self): return '*'
...
int(I(65))
42
float(F(65))
42.0
str(S('A'))
'*'
chr(I(65))
'A'
import cmath; cmath.rect(F(65), 0)
(65+0j)
ord(S('A'))
65
Issue17576 [1] proposes to call special methods for implicit converting.
I have doubts about this.
1. I afraid that this will adds places where arbitrary Python code is
unexpectedly called. For example see changeset9a61be172c23 discussed in
neighbor thread. If the "k" format code will call __int__(), Python code
can modify unpacked list argument during parsing arguments in
PyArg_ParseTuple().
2. PyLong_As*() functions already is not very consistent. Some of them
calls __int__() for argument which is not an instance of int subclass,
other accepts only instances of int subclasses. PyLong_AsVoidPtr() calls
or not calls __int__() depending on the sign of the argument.
3. We can't consistency call special methods for all types. E.g. for
strings we can't call __str__() when processing the "s" code in
PyArg_ParseTuple() because this will cause a leak.
I think that overriding special converting method in a subclass of
corresponding type should be restricted. I see two consistent and safe
possibilities:
1. Forbidden. I.e. above declarations of I, F and S classes should raise
exceptions.
2. Has no effect. I.e. both int(I(65)) and operator.index(I(65)) should
return 65.
[1] http://bugs.python.org/issue17576
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Bug? http.client assumes iso-8859-1 encoding of HTTP headers
Hi Python devs,
I am trying to download an HTML document. I get an HTTP 301 (Moved
Permanently) with a UTF-8 encoded Location header and http.client decodes
it as iso-8859-1. When there's a non-ASCII character in the redirect URL
then I can't download the document.
In client.py def parse_headers() I see the call to decode('iso-8859-1'). My
personal hack is to use whatever charset is defined in the Content-Type
HTTP header (utf8) or fall back into iso-8859-1.
At this point I am not sure where/how a fix should occur so I thought I'd
run it by you in case I should file a bug. Note that I don't use
http.client directly, but through the python-requests library.
I include some code to reproduce the problem below.
Cheers,
Hugo
-
#!/usr/bin/env python3
# Trying to replicate what wget does with a 301 redirect:
# wget --server-response
www.starbucks.com/store/158/AT/Karntnerstrasse/K%c3%a4rntnerstrasse-49-Vienna-9-1010
import http.client
import urllib.parse
s2='/store/158/AT/Karntnerstrasse/K%c3%a4rntnerstrasse-49-Vienna-9-1010'
s3='
http://www.starbucks.com/store/158/at/karntnerstrasse/k%C3%A4rntnerstrasse-49-vienna-9-1010
'
conn = http.client.HTTPConnection('www.starbucks.com')
conn.request('GET', s2)
r = conn.getresponse()
print('Location', r.headers.get('Location'))
print('Expected', urllib.parse.unquote(s3))
assert r.status == 301
assert r.headers.get('Location') == urllib.parse.unquote(s3), \
'decoded as iso-8859-1 instead of utf8'
conn = http.client.HTTPConnection('www.starbucks.com')
conn.request('GET', s3)
r = conn.getresponse()
assert r.status == 200
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug? http.client assumes iso-8859-1 encoding of HTTP headers
On Sun, Jan 5, 2014 at 2:36 AM, Hugo G. Fierro wrote:
> I am trying to download an HTML document. I get an HTTP 301 (Moved
> Permanently) with a UTF-8 encoded Location header and http.client decodes it
> as iso-8859-1. When there's a non-ASCII character in the redirect URL then I
> can't download the document.
>
> In client.py def parse_headers() I see the call to decode('iso-8859-1'). My
> personal hack is to use whatever charset is defined in the Content-Type
> HTTP header (utf8) or fall back into iso-8859-1.
>
> At this point I am not sure where/how a fix should occur so I thought I'd
> run it by you in case I should file a bug. Note that I don't use http.client
> directly, but through the python-requests library.
I'm not 100% sure, but I believe non-ASCII characters are outright
forbidden in a Location: header. It's possible that an RFC2047 tag
might be used, but my reading of RFC2616 is that that's only for text
fields, not for Location. These non-ASCII characters ought to be
percent-encoded, and anything doing otherwise is buggy.
Confirming what you're seeing with a plain socket:
>>> s=socket.socket()
>>> s.connect(("www.starbucks.com",80))
>>> s.send(b'GET
>>> /store/158/AT/Karntnerstrasse/K%c3%a4rntnerstrasse-49-Vienna-9-1010
>>> HTTP/1.1\r\nHost: www.starbucks.com\r\nAccept-Encoding: identity\r\n\r\n')
136
>>> s.recv(1024)
b'HTTP/1.1 301 Moved Permanently\r\nContent-Type: text/html;
charset=UTF-8\r\nLocation:
http://www.starbucks.com/store/158/at/karntnerstrasse/k\xc3\xa4rntnerstrasse-49-vienna-9-1010\r\n
'
I'm pretty sure that server is in violation of the spec, so all bets
are off as to what any other server will do. If you know you're
dealing with this one server, you can probably hack around this, but I
don't think it belongs in core code. Unless, of course, I'm completely
wrong about the spec, or if there's a de facto spec that lots of
servers follow, in which case maybe it would be worth doing.
ChrisA
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug? http.client assumes iso-8859-1 encoding of HTTP headers
On 2014-01-04, at 17:24 , Chris Angelico wrote:
> On Sun, Jan 5, 2014 at 2:36 AM, Hugo G. Fierro wrote:
>> I am trying to download an HTML document. I get an HTTP 301 (Moved
>> Permanently) with a UTF-8 encoded Location header and http.client decodes it
>> as iso-8859-1. When there's a non-ASCII character in the redirect URL then I
>> can't download the document.
>>
>> In client.py def parse_headers() I see the call to decode('iso-8859-1'). My
>> personal hack is to use whatever charset is defined in the Content-Type
>> HTTP header (utf8) or fall back into iso-8859-1.
>>
>> At this point I am not sure where/how a fix should occur so I thought I'd
>> run it by you in case I should file a bug. Note that I don't use http.client
>> directly, but through the python-requests library.
>
> I'm not 100% sure, but I believe non-ASCII characters are outright
> forbidden in a Location: header. It's possible that an RFC2047 tag
> might be used, but my reading of RFC2616 is that that's only for text
> fields, not for Location. These non-ASCII characters ought to be
> percent-encoded, and anything doing otherwise is buggy.
That is also my reading, the Location field’s value is defined as an
absoluteURI (RFC2616, section 14.30):
> Location = "Location" ":" absoluteURI
section 3.2.1 indicates that "absoluteURI" (and other related
concepts) are used as defined by RFC 2396 "Uniform Resource
Identifiers (URI): Generic Syntax", that is:
> absoluteURI = scheme ":" ( hier_part | opaque_part )
both "hier_part" and "opaque_part" consist of some punctuation
characters, "escaped" and "unreserved". "escaped" is %-encoded
characters which leaves "unreserved" defined as "alphanum | mark".
"mark" is more punctuation and "alphanum" is ASCII's alphanumeric
ranges.
Furthermore, although RFC 3986 moves some stuff around and renames some
production rules, it seems to have kept this limitation.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.x vs 3.x survey results
On 2014-01-02 17:54, Dan Stromberg wrote: > I put it at https://wiki.python.org/moin/2.x-vs-3.x-survey It would've been nice to see some crosstabs. Pretty much any question after Q3 is incomprehensible without splitting the respondents into sub-groups first. Of the 2.49% of people who said they've never written Python 2.x, how many of those people also said they have never written Python 3.x too? (There really is 4 categories of developers being surveyed here.) Of the 22.91% of people who said Python 3.x was a mistake, how many of them also said they have never written any Python 3.x? Of the 40% of people who said they have never written Python 3.x, how many of them also said they had dependencies keeping them on Python 2.x? Etc. -- Scott Dial [email protected] ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #19976: Argument Clinic METH_NOARGS functions now always
Probably Rietveld did not send mail, so I mention my review comments again: larry.hastings wrote: > +#ifdef __GNUC__ > +#define Py_UNUSED(name) _unused_ ## name __attribute__((unused)) > +#else > +#define Py_UNUSED(name) _unused_ ## name > +#endif > + The Intel compiler defines __GNUC__ but chokes on the __attribute__(). This works: #if defined(__GNUC__) && !defined(__INTEL_COMPILER) > +_pickle_Pickler_clear_memo(PyObject *self, PyObject *Py_UNUSED(ignored)) I'm not a native speaker, but UNUSED(ignored) reads strange to me. I would prefer UNUSED(args). Stefan Krah ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug? http.client assumes iso-8859-1 encoding of HTTP headers
Quoting Chris Angelico : I'm pretty sure that server is in violation of the spec, so all bets are off as to what any other server will do. If you know you're dealing with this one server, you can probably hack around this, but I don't think it belongs in core code. Unless, of course, I'm completely wrong about the spec, or if there's a de facto spec that lots of servers follow, in which case maybe it would be worth doing. It would be possible to support this better by using "ascii" with "surrogateescape" when receiving the redirect, and using the same for all URLs coming into http.client. This would implement a best-effort strategy at preserving the bogus URL, and still maintain the notion that URLs are text (with the other path being to also allow bytes as URLs, and always parsing Location as bytes). Regards, Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #19976: Argument Clinic METH_NOARGS functions now always
Stefan Krah, 04.01.2014 21:00: > Probably Rietveld did not send mail, so I mention my review comments again: > > larry.hastings wrote: >> +#ifdef __GNUC__ >> +#define Py_UNUSED(name) _unused_ ## name __attribute__((unused)) >> +#else >> +#define Py_UNUSED(name) _unused_ ## name >> +#endif >> + > > The Intel compiler defines __GNUC__ but chokes on the __attribute__(). > > This works: > > #if defined(__GNUC__) && !defined(__INTEL_COMPILER) We use this in Cython and according to the mailing list echo, it would seem that there are people running it through Intel's compiler as well: """ #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif """ I wonder why this works, though, given that you say Intel doesn't support "__attribute__". The only difference I can spot is the space behind it. In any case, I agree that the right way to do it is a bit more complex than in the original commit. Stefan ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #19976: Argument Clinic METH_NOARGS functions now always
Stefan Behnel wrote: > """ > #ifndef CYTHON_UNUSED > # if defined(__GNUC__) > # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && > __GNUC_MINOR__ >= 4)) > # define CYTHON_UNUSED __attribute__ ((__unused__)) > # else > # define CYTHON_UNUSED > # endif > # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) > # define CYTHON_UNUSED __attribute__ ((__unused__)) > # else > # define CYTHON_UNUSED > # endif > #endif > """ > > I wonder why this works, though, given that you say Intel doesn't support > "__attribute__". The only difference I can spot is the space behind it. You're right, icc version 12.0 supports the attribute. It must have been some earlier version that failed. Stefan Krah ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.x vs 3.x survey results
On 4 January 2014 19:21, Scott Dial wrote: > On 2014-01-02 17:54, Dan Stromberg wrote: > > I put it at https://wiki.python.org/moin/2.x-vs-3.x-survey > > It would've been nice to see some crosstabs. Pretty much any question > after Q3 is incomprehensible without splitting the respondents into > sub-groups first. > > Of the 2.49% of people who said they've never written Python 2.x, how > many of those people also said they have never written Python 3.x too? > (There really is 4 categories of developers being surveyed here.) Of the > 22.91% of people who said Python 3.x was a mistake, how many of them > also said they have never written any Python 3.x? Of the 40% of people > who said they have never written Python 3.x, how many of them also said > they had dependencies keeping them on Python 2.x? Etc. > > -- > Scott Dial > [email protected] > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/luca.sbardella%40gmail.com > Hi guys, you are my heroes but this survey is quite useless, can you include more people? I wasn't aware of it so many thousands of python users. And after that, you are well aware that Python 3 or 2 is becoming a liability, just stick with one, anyone (3) at this point. I don't want to go and learn a new language, please. Sorry for the rant ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.x vs 3.x survey results
Luca Sbardella writes: > you are my heroes but this survey is quite useless, can you include more > people? The survey cohort was self-selected from those who read the forums where it was posted. > I wasn't aware of it so many thousands of python users. That statement confuses me. Were you aware of it, or not? How did you become aware of it? > And after that, you are well aware that Python 3 or 2 is becoming a > liability, just stick with one, anyone (3) at this point. The policy of the Python core developers is quite clear, and has been for many years: Python 2 is a dead end, and Python 2.7 (released 2010-07-03, 3½ years ago) is the last Python 2. Python 2.7 is the last of the Python 2 line, there will never be new Python 2 features http://python.org/dev/peps/pep-0404/>, everyone should migrate to Python 3. That is already the Python core developers's published policy. So, to whom are you speaking here on the Python core developers' forum? > I don't want to go and learn a new language, please. Great! If you already know Python, then there is very little (certainly not “a new language”) different to move from Python 2.7 to Python 3. Enjoy! -- \ “Our task must be to free ourselves from our prison by widening | `\our circle of compassion to embrace all humanity and the whole | _o__) of nature in its beauty.” —Albert Einstein | Ben Finney ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.x vs 3.x survey results
On 1/4/14 10:42 PM, Ben Finney wrote: Luca Sbardella writes: you are my heroes but this survey is quite useless, can you include more people? The survey cohort was self-selected from those who read the forums where it was posted. I wasn't aware of it so many thousands of python users. That statement confuses me. Were you aware of it, or not? How did you become aware of it? And after that, you are well aware that Python 3 or 2 is becoming a liability, just stick with one, anyone (3) at this point. The policy of the Python core developers is quite clear, and has been for many years: Python 2 is a dead end, and Python 2.7 (released 2010-07-03, 3½ years ago) is the last Python 2. Python 2.7 is the last of the Python 2 line, there will never be new Python 2 features http://python.org/dev/peps/pep-0404/>, everyone should migrate to Python 3. That is already the Python core developers's published policy. So, to whom are you speaking here on the Python core developers' forum? I don't want to go and learn a new language, please. Great! If you already know Python, then there is very little (certainly not “a new language”) different to move from Python 2.7 to Python 3. Enjoy! I think it helps Luca and many others (including myself) if there is a reference of the difference between 2.7 and Python 3.3+. There are PEPs and books, but is there any such long list of references? If not, should we start investing in one? I know the basic one such as xrange and range, items vs iteritems, izip vs zip that sort of uniform syntax/library inclusion difference. If there is such reference available? Yeuk Hon ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.x vs 3.x survey results
On Sat, Jan 4, 2014 at 8:20 PM, John Yeuk Hon Wong wrote: > I think it helps Luca and many others (including myself) if there is a > reference of the difference between 2.7 and Python 3.3+. > There are PEPs and books, but is there any such long list of references? > > If not, should we start investing in one? I know the basic one such as > xrange and range, items vs iteritems, izip vs zip that sort of uniform > syntax/library inclusion difference. > > If there is such reference available? This isn't comprehensive, but it does cover the issues I ran into while writing a backup program (of about 7,000 lines) that runs on 2.x and 3.x, unmodified: http://stromberg.dnsalias.org/~dstromberg/Intro-to-Python/ Specifically, I mean the "Writing code to run on Python 2.x and 3.x" document. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
