[issue1145] Allow str.join to join non-string types (as per PEP 3100)
New submission from Thomas Lee:
The current implementation of str.join requires that the parameters
passed to it be string/unicode values. A suggestion to allow it to
accept parameters of any type came up in PEP 3100. Implemented for
Unicode using the attached patch.
It would be trivial to add this functionality to the old string object
too, but I haven't been following the string to unicode discussion too
closely and I'm unsure if the old string object will remain for too much
longer.
--
components: Interpreter Core
files: join-autostr.patch
messages: 55820
nosy: thomas.lee
severity: minor
status: open
title: Allow str.join to join non-string types (as per PEP 3100)
type: behavior
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1145>
__Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58102)
+++ Objects/unicodeobject.c (working copy)
@@ -5412,14 +5412,16 @@
item = PySequence_Fast_GET_ITEM(fseq, i);
/* Convert item to Unicode. */
- if (! PyUnicode_Check(item) && ! PyString_Check(item)) {
- PyErr_Format(PyExc_TypeError,
- "sequence item %zd: expected string or Unicode,"
- " %.80s found",
- i, Py_Type(item)->tp_name);
- goto onError;
+ if (!PyString_Check(item) && !PyUnicode_Check(item))
+ {
+ PyObject* sval;
+ sval = PyObject_Unicode(item);
+ item = PyUnicode_FromObject(sval);
+ Py_DECREF(sval);
}
- item = PyUnicode_FromObject(item);
+ else
+ item = PyUnicode_FromObject(item);
+
if (item == NULL)
goto onError;
/* We own a reference to item from here on. */
Index: Doc/library/stdtypes.rst
===
--- Doc/library/stdtypes.rst (revision 58102)
+++ Doc/library/stdtypes.rst (working copy)
@@ -786,8 +786,10 @@
.. method:: str.join(seq)
- Return a string which is the concatenation of the strings in the sequence *seq*.
- The separator between elements is the string providing this method.
+ Return a string which is the concatenation of the values in the sequence *seq*.
+ Non-string values in *seq* will be converted to a string using their respective
+ implementation of :meth:`__str__`. The separator between elements is the string
+ providing this method.
.. method:: str.ljust(width[, fillchar])
Index: Lib/test/test_descr.py
===
--- Lib/test/test_descr.py (revision 58102)
+++ Lib/test/test_descr.py (working copy)
@@ -3238,10 +3238,6 @@
except ValueError: pass
else: raise TestFailed("''.split('') doesn't raise ValueError")
-try: ''.join([0])
-except TypeError: pass
-else: raise TestFailed("''.join([0]) doesn't raise TypeError")
-
try: ''.rindex('5')
except ValueError: pass
else: raise TestFailed("''.rindex('5') doesn't raise ValueError")
Index: Lib/test/test_unicode.py
===
--- Lib/test/test_unicode.py (revision 58102)
+++ Lib/test/test_unicode.py (working copy)
@@ -178,6 +178,10 @@
def test_join(self):
string_tests.MixinStrUnicodeUserStringTest.test_join(self)
+class MyWrapper:
+def __init__(self, sval): self.sval = sval
+def __str__(self): return self.sval
+
# mixed arguments
self.checkequalnofix('a b c d', ' ', 'join', ['a', 'b', 'c', 'd'])
self.checkequalnofix('abcd', '', 'join', ('a', 'b', 'c', 'd'))
@@ -186,6 +190,7 @@
self.checkequalnofix('a b c d', ' ', 'join', ['a', 'b', 'c', 'd'])
self.checkequalnofix('abcd', '', 'join', ('a', 'b', 'c', 'd'))
self.checkequalnofix('w x y z', ' ', 'join', string_tests.Sequence('wxyz'))
+self.checkequalnofix('1 2 foo you', ' ', 'join', [1, 2, MyWrapper('foo'), 'you'])
def test_replace(self):
string_tests.CommonTest.test_replace(self)
Index: Lib/test/string_tests.py
===
--- Lib/test/string_tests.py (revision 58102)
+++ Lib/test/string_tests.py (working copy)
@@ -13,6 +13,7 @@
class BadSeq1(Sequence):
def __init__(self): self.seq = [7, 'hello', 123]
+def __str__(self): return '{0} {1} {2}'.format(*self.seq)
class BadSeq2(Sequence
[issue1145] Allow str.join to join non-string types (as per PEP 3100)
Thomas Lee added the comment: Oh and an example of usage: # before the patch ', '.join([str(x) for x in [1, 2, 3]]) # after the patch ', '.join([1, 2, 3]) __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1145> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1145] Allow str.join to join non-string types (as per PEP 3100)
Thomas Lee added the comment: Sure - I'll get onto that. Should have another patch up later tonight. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1145> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1145] Allow str.join to join non-string types (as per PEP 3100)
Thomas Lee added the comment:
Updated patch:
* unneeded PyUnicode_FromObject call removed (I thought this was
necessary, but the original author appears to be using it for an INCREF)
* documentation now fits into 80 chars
* return values from PyObject_Unicode and PyObject_FromObject checked
* bytes() objects found in the sequence will raise a TypeError
* removed redundant assertion and added the bytes case to test_unicode
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1145>
__Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58102)
+++ Objects/unicodeobject.c (working copy)
@@ -5412,14 +5412,20 @@
item = PySequence_Fast_GET_ITEM(fseq, i);
/* Convert item to Unicode. */
- if (! PyUnicode_Check(item) && ! PyString_Check(item)) {
- PyErr_Format(PyExc_TypeError,
- "sequence item %zd: expected string or Unicode,"
- " %.80s found",
- i, Py_Type(item)->tp_name);
- goto onError;
+ if (!PyString_Check(item) && !PyUnicode_Check(item))
+ {
+ if (PyBytes_Check(item))
+ {
+ PyErr_Format(PyExc_TypeError,
+"sequence item %d: join() will not operate on "
+"bytes objects", i);
+ goto onError;
+ }
+ item = PyObject_Unicode(item);
}
- item = PyUnicode_FromObject(item);
+ else
+ item = PyUnicode_FromObject(item);
+
if (item == NULL)
goto onError;
/* We own a reference to item from here on. */
Index: Doc/library/stdtypes.rst
===
--- Doc/library/stdtypes.rst (revision 58102)
+++ Doc/library/stdtypes.rst (working copy)
@@ -786,8 +786,11 @@
.. method:: str.join(seq)
- Return a string which is the concatenation of the strings in the sequence *seq*.
- The separator between elements is the string providing this method.
+ Return a string which is the concatenation of the values in the sequence
+ *seq*. Non-string values in *seq* will be converted to a string using their
+ respective ``str()`` value. If there are any :class:`bytes` objects in
+ *seq*, a :exc:`TypeError` will be raised. The separator between elements is
+ the string providing this method.
.. method:: str.ljust(width[, fillchar])
Index: Lib/test/test_descr.py
===
--- Lib/test/test_descr.py (revision 58102)
+++ Lib/test/test_descr.py (working copy)
@@ -3238,10 +3238,6 @@
except ValueError: pass
else: raise TestFailed("''.split('') doesn't raise ValueError")
-try: ''.join([0])
-except TypeError: pass
-else: raise TestFailed("''.join([0]) doesn't raise TypeError")
-
try: ''.rindex('5')
except ValueError: pass
else: raise TestFailed("''.rindex('5') doesn't raise ValueError")
Index: Lib/test/test_unicode.py
===
--- Lib/test/test_unicode.py (revision 58102)
+++ Lib/test/test_unicode.py (working copy)
@@ -178,6 +178,10 @@
def test_join(self):
string_tests.MixinStrUnicodeUserStringTest.test_join(self)
+class MyWrapper:
+def __init__(self, sval): self.sval = sval
+def __str__(self): return self.sval
+
# mixed arguments
self.checkequalnofix('a b c d', ' ', 'join', ['a', 'b', 'c', 'd'])
self.checkequalnofix('abcd', '', 'join', ('a', 'b', 'c', 'd'))
@@ -186,6 +190,8 @@
self.checkequalnofix('a b c d', ' ', 'join', ['a', 'b', 'c', 'd'])
self.checkequalnofix('abcd', '', 'join', ('a', 'b', 'c', 'd'))
self.checkequalnofix('w x y z', ' ', 'join', string_tests.Sequence('wxyz'))
+self.checkequalnofix('1 2 foo', ' ', 'join', [1, 2, MyWrapper('foo')])
+self.checkraises(TypeError, ' ', 'join', [1, 2, 3, bytes()])
def test_replace(self):
string_tests.CommonTest.test_replace(self)
Index: Lib/test/string_tests.py
===
--- Lib/test/string_tests.py (revision 58102)
+++ Lib/test/string_tests.py (working copy)
@@ -13,6 +13,7 @@
class BadSeq1(Sequence):
def __init__(self): self.seq = [7, 'hello', 123]
+def __str__(self): return '{0} {1} {2}'.format(*self.seq)
class BadSeq2(S
[issue1145] Allow str.join to join non-string types (as per PEP 3100)
Thomas Lee added the comment: Is there anything else you need from me for this one Guido? __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1145> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1249] PEP 3137 patch: make PyBytes/PyUnicode ==/!= comparisons return False
New submission from Thomas Lee:
Initial patch attached.
--
components: Interpreter Core
files: bytes-unicode-return-false-r1.patch
messages: 56284
nosy: thomas.lee
severity: normal
status: open
title: PEP 3137 patch: make PyBytes/PyUnicode ==/!= comparisons return False
type: rfe
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1249>
__Index: Objects/bytesobject.c
===
--- Objects/bytesobject.c (revision 58387)
+++ Objects/bytesobject.c (working copy)
@@ -964,8 +964,8 @@
error, even if the comparison is for equality. */
if (PyObject_IsInstance(self, (PyObject*)&PyUnicode_Type) ||
PyObject_IsInstance(other, (PyObject*)&PyUnicode_Type)) {
-PyErr_SetString(PyExc_TypeError, "can't compare bytes and str");
-return NULL;
+Py_INCREF(Py_False);
+return Py_False;
}
self_size = _getbuffer(self, &self_bytes);
Index: Lib/test/test_bytes.py
===
--- Lib/test/test_bytes.py (revision 58387)
+++ Lib/test/test_bytes.py (working copy)
@@ -132,10 +132,10 @@
# Bytes can't be compared to Unicode!
# Test this for all expected byte orders and Unicode character sizes
-self.assertRaises(TypeError, lambda: b"\0a\0b\0c" == "abc")
-self.assertRaises(TypeError, lambda: b"\0\0\0a\0\0\0b\0\0\0c" == "abc")
-self.assertRaises(TypeError, lambda: b"a\0b\0c\0" == "abc")
-self.assertRaises(TypeError, lambda: b"a\0\0\0b\0\0\0c\0\0\0" == "abc")
+self.assertEqual(b"\0a\0b\0c" == "abc", False)
+self.assertEqual(b"\0\0\0a\0\0\0b\0\0\0c" == "abc", False)
+self.assertEqual(b"a\0b\0c\0" == "abc", False)
+self.assertEqual(b"a\0\0\0b\0\0\0c\0\0\0" == "abc", False)
def test_nohash(self):
self.assertRaises(TypeError, hash, bytes())
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1249] PEP 3137 patch: PyBytes/PyUnicode comparisons
Thomas Lee added the comment:
Revised patch - originally misinterpreted what was required here.
bytes() == str() now returns False, bytes() != str() now returns True.
--
title: PEP 3137 patch: make PyBytes/PyUnicode ==/!= comparisons return False ->
PEP 3137 patch: PyBytes/PyUnicode comparisons
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1249>
__Index: Objects/bytesobject.c
===
--- Objects/bytesobject.c (revision 58389)
+++ Objects/bytesobject.c (working copy)
@@ -964,8 +964,9 @@
error, even if the comparison is for equality. */
if (PyObject_IsInstance(self, (PyObject*)&PyUnicode_Type) ||
PyObject_IsInstance(other, (PyObject*)&PyUnicode_Type)) {
-PyErr_SetString(PyExc_TypeError, "can't compare bytes and str");
-return NULL;
+PyErr_Clear();
+Py_INCREF(Py_NotImplemented);
+return Py_NotImplemented;
}
self_size = _getbuffer(self, &self_bytes);
Index: Lib/test/test_bytes.py
===
--- Lib/test/test_bytes.py (revision 58389)
+++ Lib/test/test_bytes.py (working copy)
@@ -130,12 +130,14 @@
self.assertEqual(str8("abc") < b"ab", False)
self.assertEqual(str8("abc") <= b"ab", False)
-# Bytes can't be compared to Unicode!
+# Byte comparisons with unicode should always fail!
# Test this for all expected byte orders and Unicode character sizes
-self.assertRaises(TypeError, lambda: b"\0a\0b\0c" == "abc")
-self.assertRaises(TypeError, lambda: b"\0\0\0a\0\0\0b\0\0\0c" == "abc")
-self.assertRaises(TypeError, lambda: b"a\0b\0c\0" == "abc")
-self.assertRaises(TypeError, lambda: b"a\0\0\0b\0\0\0c\0\0\0" == "abc")
+self.assertEqual(b"\0a\0b\0c" == "abc", False)
+self.assertEqual(b"\0\0\0a\0\0\0b\0\0\0c" == "abc", False)
+self.assertEqual(b"a\0b\0c\0" == "abc", False)
+self.assertEqual(b"a\0\0\0b\0\0\0c\0\0\0" == "abc", False)
+self.assertEqual(bytes() == str(), False)
+self.assertEqual(bytes() != str(), True)
def test_nohash(self):
self.assertRaises(TypeError, hash, bytes())
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1263] PEP 3137 patch - str8/str comparison should return false
New submission from Thomas Lee:
The main patch - while exactly what is needed to make str8/str equality
checks return False - breaks a bunch of tests due to PyString_* still
being used elsewhere when it should be using PyUnicode.
The second patch modifies structmember.c to use PyUnicode_* where it was
previously using PyString_*, which fixes the first problem I stumbled
across in trying to get test_unicode to run.
Unfortunately, similar errors are present in Python/codecs.c and other
places (maybe Python/modsupport.c too? not 100% sure yet) - these still
need to be fixed!
--
components: Interpreter Core
files: unicode-string-eq-false-r2.patch
messages: 56343
nosy: thomas.lee
severity: normal
status: open
title: PEP 3137 patch - str8/str comparison should return false
type: rfe
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1263>
__Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58389)
+++ Objects/unicodeobject.c (working copy)
@@ -6191,16 +6191,6 @@
if (PyUnicode_Check(left) && PyUnicode_Check(right))
return unicode_compare((PyUnicodeObject *)left,
(PyUnicodeObject *)right);
-if ((PyString_Check(left) && PyUnicode_Check(right)) ||
-(PyUnicode_Check(left) && PyString_Check(right))) {
-if (PyUnicode_Check(left))
-left = _PyUnicode_AsDefaultEncodedString(left, NULL);
-if (PyUnicode_Check(right))
-right = _PyUnicode_AsDefaultEncodedString(right, NULL);
-assert(PyString_Check(left));
-assert(PyString_Check(right));
-return PyObject_Compare(left, right);
-}
PyErr_Format(PyExc_TypeError,
"Can't compare %.100s and %.100s",
left->ob_type->tp_name,
Index: Lib/stringprep.py
===
--- Lib/stringprep.py (revision 58389)
+++ Lib/stringprep.py (working copy)
@@ -5,6 +5,8 @@
and mappings, for which a mapping function is provided.
"""
+import sys
+
from unicodedata import ucd_3_2_0 as unicodedata
assert unicodedata.unidata_version == '3.2.0'
Index: Lib/test/test_unicode.py
===
--- Lib/test/test_unicode.py (revision 58389)
+++ Lib/test/test_unicode.py (working copy)
@@ -200,6 +200,10 @@
self.checkequalnofix('[EMAIL PROTECTED]', 'one!two!three!', 'replace', '!', '@', 1)
self.assertRaises(TypeError, 'replace'.replace, "r", 42)
+def test_str8_comparison(self):
+self.assertEqual('abc' == str8('abc'), False)
+self.assertEqual('abc' != str8('abc'), True)
+
def test_comparison(self):
# Comparisons:
self.assertEqual('abc', 'abc')
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1263] PEP 3137 patch - str8/str comparison should return false
Changes by Thomas Lee: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1263> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1263] PEP 3137 patch - str8/str comparison should return false
Thomas Lee added the comment:
Oops - use unicode-string-eq-false-r3.patch, not
unicode-string-eq-false-r2.patch.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1263>
__Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58389)
+++ Objects/unicodeobject.c (working copy)
@@ -6191,16 +6191,6 @@
if (PyUnicode_Check(left) && PyUnicode_Check(right))
return unicode_compare((PyUnicodeObject *)left,
(PyUnicodeObject *)right);
-if ((PyString_Check(left) && PyUnicode_Check(right)) ||
-(PyUnicode_Check(left) && PyString_Check(right))) {
-if (PyUnicode_Check(left))
-left = _PyUnicode_AsDefaultEncodedString(left, NULL);
-if (PyUnicode_Check(right))
-right = _PyUnicode_AsDefaultEncodedString(right, NULL);
-assert(PyString_Check(left));
-assert(PyString_Check(right));
-return PyObject_Compare(left, right);
-}
PyErr_Format(PyExc_TypeError,
"Can't compare %.100s and %.100s",
left->ob_type->tp_name,
Index: Lib/test/test_unicode.py
===
--- Lib/test/test_unicode.py (revision 58389)
+++ Lib/test/test_unicode.py (working copy)
@@ -200,6 +200,10 @@
self.checkequalnofix('[EMAIL PROTECTED]', 'one!two!three!', 'replace', '!', '@', 1)
self.assertRaises(TypeError, 'replace'.replace, "r", 42)
+def test_str8_comparison(self):
+self.assertEqual('abc' == str8('abc'), False)
+self.assertEqual('abc' != str8('abc'), True)
+
def test_comparison(self):
# Comparisons:
self.assertEqual('abc', 'abc')
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1263] PEP 3137 patch - str8/str comparison should return false
Thomas Lee added the comment:
Hack to make Python/codecs.c use Unicode strings internally. I recognize
the way I have fixed it here is probably not ideal (basically ripped out
PyString_*, replaced with a PyMem_Malloc/PyMem_Free call) but it fixes
10-12 tests that were failing with my earlier changes. If anybody can
recommend a "nice" way to fix this, I'm all ears.
The following still fail for me with this patch applied:
-- test_compile
This is due to PyString_*/PyUnicode_*/PyBytes_* confusion in the
assembler struct (specifically: a_lnotab and a_bytecode) in
Python/compile.c - tried replacing PyString_* calls with PyBytes_*
calls, but this raises a TypeError because PyBytes is not hashable ...
not sure what exactly is causing this.
-- test_ctypes
Looks like a simple case of ctypes using str8 instead of str. Appears to
be an easy fix.
-- test_modulefinder
Failing because str8 >= str is now an invalid operation
-- test_set
This test needs some love.
-- test_sqlite
Not sure what's going on here.
-- test_str
This one is a little tricky: str8/str with __str__/__unicode__ ... how
is this test supposed to behave with the fix in this patch?
-- test_struct
"unpack/pack not transitive" - what does that mean?
-- test_subprocess
Like modulefinder, this is probably just due to the use of str8 over str
internally in the subprocess module. Likely to be an easy fix.
The following tests fail for me irrespective of whether or not I have r4
of my patch applied:
-- test_doctest
-- test_email
-- test_nis
-- test_pty
If anybody can give this new patch a try and let me know the result it
would be much appreciated.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1263>
__Index: Python/codecs.c
===
--- Python/codecs.c (revision 58468)
+++ Python/codecs.c (working copy)
@@ -55,16 +55,15 @@
size_t len = strlen(string);
char *p;
PyObject *v;
-
+
if (len > PY_SSIZE_T_MAX) {
PyErr_SetString(PyExc_OverflowError, "string is too large");
return NULL;
}
-v = PyString_FromStringAndSize(NULL, len);
-if (v == NULL)
- return NULL;
-p = PyString_AS_STRING(v);
+p = PyMem_Malloc(len + 1);
+if (p == NULL)
+return NULL;
for (i = 0; i < len; i++) {
register char ch = string[i];
if (ch == ' ')
@@ -73,6 +72,11 @@
ch = tolower(Py_CHARMASK(ch));
p[i] = ch;
}
+p[i] = '\0';
+v = PyUnicode_FromString(p);
+if (v == NULL)
+return NULL;
+PyMem_Free(p);
return v;
}
@@ -112,7 +116,7 @@
v = normalizestring(encoding);
if (v == NULL)
goto onError;
-PyString_InternInPlace(&v);
+PyUnicode_InternInPlace(&v);
/* First, try to lookup the name in the registry dictionary */
result = PyDict_GetItem(interp->codec_search_cache, v);
@@ -193,7 +197,7 @@
if (errors) {
PyObject *v;
- v = PyString_FromString(errors);
+ v = PyUnicode_FromString(errors);
if (v == NULL) {
Py_DECREF(args);
return NULL;
Index: Python/structmember.c
===
--- Python/structmember.c (revision 58468)
+++ Python/structmember.c (working copy)
@@ -51,13 +51,13 @@
v = Py_None;
}
else
- v = PyString_FromString(*(char**)addr);
+ v = PyUnicode_FromString(*(char**)addr);
break;
case T_STRING_INPLACE:
- v = PyString_FromString((char*)addr);
+ v = PyUnicode_FromString((char*)addr);
break;
case T_CHAR:
- v = PyString_FromStringAndSize((char*)addr, 1);
+ v = PyUnicode_FromStringAndSize((char*)addr, 1);
break;
case T_OBJECT:
v = *(PyObject **)addr;
@@ -225,8 +225,8 @@
Py_XDECREF(oldv);
break;
case T_CHAR:
- if (PyString_Check(v) && PyString_Size(v) == 1) {
- *(char*)addr = PyString_AsString(v)[0];
+ if (PyUnicode_Check(v) && PyUnicode_GetSize(v) == 1) {
+ *(char*)addr = PyUnicode_AsString(v)[0];
}
else {
PyErr_BadArgument();
Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58468)
+++ Objects/unicodeobject.c (working copy)
@@ -6224,16 +6224,6 @@
if (PyUnicode_Check(left) && PyUnicode_Check(right))
return unicode_compare((PyUnicodeObject *)left,
(PyUnicodeObject *)right);
-if ((PyString_Check(left) && PyUnicode_Check(right)) ||
-(PyUnicode_Check(left) && PyString_Check(right))) {
-if (PyUnicode_Check(left))
-left = _PyUnicode_AsDefaultEncodedString(left, NULL);
-if (PyUnicode_Check(right))
-right = _PyUnicode_AsDefaultEncodedString(right, NULL);
-assert(PyString_Check(left));
-assert(PyString_Check(right));
-
[issue1810] Partial AST compile() patch
New submission from Thomas Lee: This patch against HEAD provides the inverse operations to all the ast2obj_* functions in Python/Python-ast.c: effectively, this allows conversion to & from a PyObject representation of a Python AST. Additionally, it updates the compile() builtin to allow it to compile Python ASTs to bytecode. The patch seems to work for most simple cases, but crashes out with a segfault when trying to compile functions for some reason. -- components: Interpreter Core files: ast-r01.patch messages: 59764 nosy: thomas.lee severity: normal status: open title: Partial AST compile() patch versions: Python 2.6 Added file: http://bugs.python.org/file9134/ast-r01.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1810> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1810] Partial AST compile() patch
Thomas Lee added the comment: Attaching a sample program to demonstrate the crash. Added file: http://bugs.python.org/file9135/ast.py __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1810> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1810] AST compile() patch
Thomas Lee added the comment: I knew it would be a simple one for somebody who knew what to look for :) Thanks Georg! r02 is the updated patch. Changing the title of the tracker issue to reflect that this *should* be a complete patch now. Any further recommendations? -- title: Partial AST compile() patch -> AST compile() patch Added file: http://bugs.python.org/file9147/ast-r02.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1810> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1772] popen fails when there are two or more pathnames/parameters with spaces
Thomas Lee added the comment: Do you have a code sample that reproduces this issue? Which popen module and/or function are you using to execute this command? -- nosy: +thomas.lee __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1772> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Thomas Lee added the comment: If you're referring to an 'append' mode for bz2file objects, it may be a limitation of the underlying library: my version of bzlib.h only provides BZ2_bzWriteOpen and BZ2_bzReadOpen - it's not immediately clear how you would open a BZ2File in append mode looking at this API. It may be possible to implement r/w/a using the lower-level bzCompress/bzDecompress functions, but I doubt that's going to happen unless somebody (such as yourself? :)) cares deeply about this. -- nosy: +thomas.lee __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1625> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1810] AST compile() patch
Thomas Lee <[EMAIL PROTECTED]> added the comment: Updating the patch to apply cleanly against HEAD. Added file: http://bugs.python.org/file9672/ast-r03.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1810> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2725] Handle ASDLSyntaxErrors gracefully
New submission from Thomas Lee <[EMAIL PROTECTED]>: The current code in Parser/asdl.py doesn't handle syntax errors very well. A minor problem to be sure, since the net result is the same (i.e. build fails), but the error message being displayed is entirely unhelpful to the developer. The attached patch will display the error message as was seemingly intended. This probably affects earlier versions (including python-trunk), but I haven't tested. -- components: Build files: fix-asdl-errors.patch keywords: patch messages: 65999 nosy: thomas.lee severity: normal status: open title: Handle ASDLSyntaxErrors gracefully type: behavior versions: Python 3.0 Added file: http://bugs.python.org/file10144/fix-asdl-errors.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2725> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1346238] A constant folding optimization pass for the AST
Thomas Lee <[EMAIL PROTECTED]> added the comment: I'm working on the AST optimization code for 2.7 (?) in the tlee-ast-optimize branch. I've since adopted some of the ideas from this patch, but I'll take another look when I get a chance. The folding operation is already mostly in-place. -- nosy: +thomas.lee _ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1346238> _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4113] functools.partial(), no __name__; wrong __doc__
Thomas Lee <[EMAIL PROTECTED]> added the comment: Here's a patch against the 2.x trunk. Basically just dispatches reads of a partial's __name__ and __doc__ attributes to the underlying function object. -- keywords: +patch nosy: +thomas.lee Added file: http://bugs.python.org/file11831/issue4113-01.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4113> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4147] xml.dom.minidom toprettyxml: omit whitespace for text-only elements
New submission from Thomas Lee <[EMAIL PROTECTED]>: For XML elements containing only text data, it would be nice if toprettyxml could omit the whitespace it normally injects before & after the text, e.g. Bob Becomes: Bob >From what I understand the handling of whitespace within XML elements is application-defined, so I'm classifying this as a nice-to-have feature. However it should be noted that in our particular case, the existing behaviour caused a few problems with a third-party system which treated whitespace as being significant. -- components: Library (Lib), XML files: minidom-toprettyxml-01.patch keywords: patch messages: 74978 nosy: thomas.lee severity: normal status: open title: xml.dom.minidom toprettyxml: omit whitespace for text-only elements type: feature request versions: Python 2.7 Added file: http://bugs.python.org/file11832/minidom-toprettyxml-01.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4147> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4113] functools.partial(), no __name__; wrong __doc__
Thomas Lee <[EMAIL PROTECTED]> added the comment: I actually agree with the sentiment Georg. Would it instead be useful to maybe provide a __repr__ implementation that describes the state of the partially applied function? I guess this is an entirely different issue, maybe one for python-ideas. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4113> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4264] Patch: optimize code to use LIST_APPEND instead of calling list.append
Thomas Lee <[EMAIL PROTECTED]> added the comment: Neal said: > I was surprised to see the change to the AST, but I understand why you did it. The problems David ran into here sound like an argument for arbitrary AST annotations -- an idea that I was toying with around the time "Const" was introduced. That way the optimizer could provide "hints" rather than explicitly manipulating the AST in certain cases. The compiler could then look for those hints and generate code accordingly. For example, in place of the Const node, we might have a "const" annotation that's applied to the top-level expression. I think I went with the Const node because it was less work, but I don't remember the details. I'll try to dig up the appropriate emails if I haven't lost them. David said: > Fixed, I think. The original code appears to be somewhat > inconsistent between files in its uses of spaces/tabs, ... Yes, they are inconsistent with tabs/spaces. And it sucks. :) > The thing that struck me as ugly was the idea that there's one > object (the optimizer) that knows everything about all > optimization operations. That's right, but this is no different from the compiler. Conversely, a visitor pattern would probably work for both the optimizer and the compiler. Having said that, I couldn't justify making the AST optimizer a huge departure from the rest of the code base for the sake of design purity. I'm not even really sure that it's "design purity" when you do things inconsistently from one component to the next. Obviously if there's another sufficiently good argument for the visitor approach, I'm all ears. But again, if we do that I think we should do it for the compiler as well. I'm not sure how much support such a change would have. -- nosy: +thomas.lee ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4264> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Dependencies of graminit.h are not rebuilt when the file is regenerated
New submission from Thomas Lee <[EMAIL PROTECTED]>: It's important that dependencies of grammar.h get rebuilt if graminit.h is regenerated (e.g. the Grammar is modified). If these dependencies do not get rebuilt, the constants associated with each type of parse node will have inconsistent values between the different intermediate files. The net result is that a program afflicted by this might build without errors, but then crash unexpectedly at runtime due to the inconsistent constant values. The patch is quite simple and ensures that all files that currently depend on graminit.h are rebuilt if it changes. It also removes an unnecessary #include from Python/future.c. I believe a similar situation might occur with Python-ast.h and the *_kind enumerations, but have yet to run into such a specific issue. I'll post a separate patch if I do find this to be a problem. -- components: Build files: graminit-dependencies.patch keywords: patch messages: 76010 nosy: thomas.lee severity: normal status: open title: Dependencies of graminit.h are not rebuilt when the file is regenerated type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file12046/graminit-dependencies.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Dependencies of graminit.h are not rebuilt when the file is regenerated
Thomas Lee <[EMAIL PROTECTED]> added the comment: Updating affected versions. Probably affects 3.x too. -- versions: +Python 2.7 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Dependencies of graminit.h are not rebuilt when the file is regenerated
Thomas Lee <[EMAIL PROTECTED]> added the comment: A deeper issue here is that Parser/parsetok.c has a dependency on graminit.h. The problem is that Parser/parsetok.c is a part of the Parser/pgen program which is actually being used to *generate* graminit.h in the first place. This breaks Python whenever syntax is added to or removed from Grammar/Grammar in such a way that the value of encoding_decl changes, because the value of encoding_decl used by pgen is different to the value used to build python itself. A simple work around for those wishing to change the syntax is a "make; make clean; make". It'd obviously be nice if the build were fixed, though. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee <[EMAIL PROTECTED]> added the comment: I mean pgen's dependency on graminit.h, not the parser's. :) ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee <[EMAIL PROTECTED]> added the comment: okay, so it turns out that putting rules in Grammar/Grammar before the top-level non-terminals breaks things in a different way. Attached is another (hopefully final and working) patch. Added file: http://bugs.python.org/file12185/build-evilness-fix-02.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee <[EMAIL PROTECTED]> added the comment: Here's a new patch that I believe should fix this issue. It modifies Makefile.pre.in to include a few additional dependency declarations for source files that depend on Include/graminit.h and Include/Python-ast.h, as well as moving encoding_decl to the top of Grammar/Grammar. This should ensure that changes to the syntax and/or AST nodes will not cause cryptic errors. Longer term, it would be great to remove the parser's dependency on graminit.h to ensure this sort of problem is never an issue. -- title: Dependencies of graminit.h are not rebuilt when the file is regenerated -> Circular dependency causes SystemError when adding new syntax Added file: http://bugs.python.org/file12184/build-evilness-fix.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee <[EMAIL PROTECTED]> added the comment: Thanks for the review Brett, apologies for the mess. I'm attaching two new patches -- one for review, the other containing the generated files. These differ very slightly from the original patch -- mainly just removing some stuff I don't think is necessary. You're probably aware of this, but it's important for the changes to the generated files to be checked in too -- likewise for testing the patch. Added file: http://bugs.python.org/file12208/build-evilness-fix-03.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee <[EMAIL PROTECTED]> added the comment: And here's the patch for review. Added file: http://bugs.python.org/file12209/build-evilness-fix-03-review.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1615] descriptor protocol bug
Thomas Lee <[EMAIL PROTECTED]> added the comment: Related reading from a few years back: http://coding.derkeiler.com/Archive/Python/comp.lang.python/2005-05/msg03829.html -- nosy: +thomas.lee ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1615> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee added the comment: Brett, any feedback on this one? ___ Python tracker <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4347] Circular dependency causes SystemError when adding new syntax
Thomas Lee added the comment: This would appear to be another build quirk: Lib/symbol.py needs to be regenerated if Grammar/Grammar changes. Brett, do you think it would be okay for this file to be generated automatically as part of the build process? I can't think of any good reason why this should continue to be a manual task. Lib/token.py is in a similar boat, except its dependency is on Include/token.h Whatever the decision, I'll provide another review/functional patch pair. ___ Python tracker <http://bugs.python.org/issue4347> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15887] urlencode should accept iterables of pairs
Thomas Lee added the comment: Working on a patch for this. Should be ready for review shortly. -- nosy: +thomaslee ___ Python tracker <http://bugs.python.org/issue15887> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15887] urlencode should accept iterables of pairs
Thomas Lee added the comment: Patch attached. Terry, you'll notice I wrap the main part of the urlencode function in a big try/except block to catch TypeErrors & ValueErrors. I'm a bit concerned that doing this may misreport the true underlying error in the event we screw up values/types in a way that doesn't relate to the 'query' argument, but I think the code seems to do a pretty good job of ensuring that won't happen -- so that fear may be unfounded. You'll see I reworked the documentation quite a bit. The wording was a bit odd in places, and I called out a few things that may not be obvious at a glance. Hopefully it seems like an improvement. :) -- keywords: +patch Added file: http://bugs.python.org/file27198/issue-15887-01.patch ___ Python tracker <http://bugs.python.org/issue15887> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15893] Py_FrozenMain() resource leak and missing malloc checks
Thomas Lee added the comment: Patch against hg tip attached. -- keywords: +patch nosy: +thomaslee Added file: http://bugs.python.org/file27202/issue-15893-01.patch ___ Python tracker <http://bugs.python.org/issue15893> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15991] BaseHTTPServer with ThreadingMixIn serving wrong data sometimes
Thomas Lee added the comment: I can reproduce this on Debian Wheezy. Both with the system Python (2.7.3rc2) and the 2.7 branch built from source. -- nosy: +thomaslee ___ Python tracker <http://bugs.python.org/issue15991> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13028] python wastes linux users time by checking for dylib on each dynamic library load
Thomas Lee added the comment: I know this is an old-ish issue, but I can't reproduce anything like this on Debian Wheezy with either Python 2.7 or tip (3.x). I think we need more details of what you're trying to do here Roger. 1. What exactly did you do to reproduce the strace output below? 2. What flavor of Linux did you attempt this on? Further, a more detailed description of what you expected to happen vs. what actually happened would be a big help here. As Victor says, .dylib shouldn't appear in strace output on Linux at all (and my testing seems to confirm this). -- nosy: +thomaslee ___ Python tracker <http://bugs.python.org/issue13028> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16061] performance regression in string replace for 3.3
Thomas Lee added the comment:
My results aren't quite as dramatic as yours, but there does appear to be a
regression:
$ ./python -V
Python 2.7.3+
$ ./python -m timeit -s "s = 'b'*1000" "s.replace('b', 'a')"
10 loops, best of 3: 16.5 usec per loop
$ ./python -V
Python 3.3.0rc3+
$ ./python -m timeit -s "s = 'b'*1000" "s.replace('b', 'a')"
1 loops, best of 3: 22.7 usec per loop
--
nosy: +thomaslee
___
Python tracker
<http://bugs.python.org/issue16061>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16062] Socket closed prematurely in httplib for https
Thomas Lee added the comment: Thanks ABR. You may be better off raising a ticket against requests (https://github.com/kennethreitz/requests). I'm assuming what you want to happen here is for the session.post() call to return the 401 response without raising an exception. Perfectly reasonable, but to the best of my knowledge the "requests" library isn't in Python's standard library -- and so this is not the place to get it fixed :) -- nosy: +thomaslee ___ Python tracker <http://bugs.python.org/issue16062> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14886] json C vs pure-python implementation difference
Thomas Lee added the comment: FWIW, I think Mark's right here. I'm +1 on the implementations being consistent. Seems like a potentially nasty surprise if you move from one implementation to the other and, lacking awareness of this quirk, design your algorithm around semantics. I think this was Mark's original point. If the json API doesn't care how the type check is performed, then we get a (probably very small :)) win from the type(o) in (list, tuple) for the Python impl in addition to bringing consistency to the two implementations. -- nosy: +thomaslee ___ Python tracker <http://bugs.python.org/issue14886> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
