[issue1032] Improve the hackish runtime_library_dirs support for gcc
New submission from Alexandre Vassalotti: In distutils.unixccompiler, there is a hack to passing correctly the -R option to gcc (and a few other compilers). However, there is a small bug that causes gcc to not be detected correctly if the compiler name does not start with "gcc" (e.g., "i486-linux-gnu-gcc", or "ccache gcc"). -- components: Distutils messages: 55317 nosy: alexandre.vassalotti severity: minor status: open title: Improve the hackish runtime_library_dirs support for gcc type: compile error __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1032> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1032] Improve the hackish runtime_library_dirs support for gcc
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1032> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1029] py3k: io.StringIO.getvalue() returns \r\n
Alexandre Vassalotti added the comment: As far as I know, StringIO should not do any string transformations. >From PEP-3116 "New I/O", last paragraph of the section "Text I/O": > Another implementation, StringIO, creates a file-like TextIO > implementation without an underlying Buffered I/O object. [...] It > does not support encodings or newline translations; you always read > back exactly the characters you wrote. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1029> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1029] py3k: io.StringIO.getvalue() returns \r\n
Alexandre Vassalotti added the comment:
> That's why the current behaviour is not correct: When I write('\n'),
> getvalue() currently returns '\r\n'.
Oh, I missed your example in your initial message. So yes, I agree that
StringIO shouldn't translate newlines like that. I attached a patch that
should fix the bug.
However, I would favor removing the "newline" keyword argument, instead.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1029>
__Index: Lib/io.py
===
--- Lib/io.py (revision 57506)
+++ Lib/io.py (working copy)
@@ -1367,6 +1367,7 @@
initial_value = str(initial_value)
self.write(initial_value)
self.seek(0)
+self._writetranslate = False
def getvalue(self):
self.flush()
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1032] Improve the hackish runtime_library_dirs support for gcc
Alexandre Vassalotti added the comment: > The patch is incorrect since find returns -1 on failure. Oops, that is right. I attached the corrected patch. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1032> __Index: Lib/distutils/unixccompiler.py === --- Lib/distutils/unixccompiler.py (revision 57274) +++ Lib/distutils/unixccompiler.py (working copy) @@ -282,7 +282,7 @@ return "+s -L" + dir elif sys.platform[:7] == "irix646" or sys.platform[:6] == "osf1V5": return ["-rpath", dir] -elif compiler[:3] == "gcc" or compiler[:3] == "g++": +elif "gcc" in compiler or "g++" in compiler: return "-Wl,-R" + dir else: return "-R" + dir ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1247] PEP 3137 patch (repr, names, parser)
Alexandre Vassalotti added the comment:
I am not sure how the parser work, so I can't comment if your change to
it is correct. It would probably a better idea to keep it for later,
when the semantic differences between str8 and bytes objects will be
resolved.
I think changing b'' to buffer(b''), in the tests, is not a good idea.
These tests will need to be changed again when bytes() will be changed
to str8().
Your changes to PyString and PyBytes look good to me. As I said on the
mailing list, I think the for-loops in bytes_repr() should be changed to:
while (*quote_prefix)
*p++ = *quote_prefix++;
I attached updated patches with the above change. I also replaced
bytes() for buffer() in PyBytes's docstrings.
--
nosy: +alexandre.vassalotti
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1247>
__Index: Objects/bytesobject.c
===
--- Objects/bytesobject.c (revision 58376)
+++ Objects/bytesobject.c (working copy)
@@ -395,7 +395,7 @@
if (i < 0)
i += Py_Size(self);
if (i < 0 || i >= Py_Size(self)) {
-PyErr_SetString(PyExc_IndexError, "bytes index out of range");
+PyErr_SetString(PyExc_IndexError, "buffer index out of range");
return NULL;
}
return PyInt_FromLong((unsigned char)(self->ob_bytes[i]));
@@ -414,7 +414,7 @@
i += PyBytes_GET_SIZE(self);
if (i < 0 || i >= Py_Size(self)) {
-PyErr_SetString(PyExc_IndexError, "bytes index out of range");
+PyErr_SetString(PyExc_IndexError, "buffer index out of range");
return NULL;
}
return PyInt_FromLong((unsigned char)(self->ob_bytes[i]));
@@ -451,7 +451,7 @@
}
}
else {
-PyErr_SetString(PyExc_TypeError, "bytes indices must be integers");
+PyErr_SetString(PyExc_TypeError, "buffer indices must be integers");
return NULL;
}
}
@@ -551,7 +551,7 @@
i += Py_Size(self);
if (i < 0 || i >= Py_Size(self)) {
-PyErr_SetString(PyExc_IndexError, "bytes index out of range");
+PyErr_SetString(PyExc_IndexError, "buffer index out of range");
return -1;
}
@@ -587,7 +587,7 @@
i += PyBytes_GET_SIZE(self);
if (i < 0 || i >= Py_Size(self)) {
-PyErr_SetString(PyExc_IndexError, "bytes index out of range");
+PyErr_SetString(PyExc_IndexError, "buffer index out of range");
return -1;
}
@@ -619,7 +619,7 @@
}
}
else {
-PyErr_SetString(PyExc_TypeError, "bytes indices must be integer");
+PyErr_SetString(PyExc_TypeError, "buffer indices must be integer");
return -1;
}
@@ -889,11 +889,14 @@
bytes_repr(PyBytesObject *self)
{
static const char *hexdigits = "0123456789abcdef";
-size_t newsize = 3 + 4 * Py_Size(self);
+const char *quote_prefix = "buffer(b'";
+const char *quote_postfix = "')";
+/* 9 prefix + 2 postfix */
+size_t newsize = 11 + 4 * Py_Size(self);
PyObject *v;
-if (newsize > PY_SSIZE_T_MAX || newsize / 4 != Py_Size(self)) {
+if (newsize > PY_SSIZE_T_MAX || (newsize-11) / 4 != Py_Size(self)) {
PyErr_SetString(PyExc_OverflowError,
-"bytes object is too large to make repr");
+"buffer object is too large to make repr");
return NULL;
}
v = PyUnicode_FromUnicode(NULL, newsize);
@@ -904,17 +907,17 @@
register Py_ssize_t i;
register Py_UNICODE c;
register Py_UNICODE *p;
-int quote = '\'';
p = PyUnicode_AS_UNICODE(v);
-*p++ = 'b';
-*p++ = quote;
+while (*quote_prefix)
+*p++ = *quote_prefix++;
+
for (i = 0; i < Py_Size(self); i++) {
/* There's at least enough room for a hex escape
and a closing quote. */
assert(newsize - (p - PyUnicode_AS_UNICODE(v)) >= 5);
c = self->ob_bytes[i];
-if (c == quote || c == '\\')
+if (c == '\'' || c == '\\')
*p++ = '\\', *p++ = c;
else if (c == '\t')
*p++ = '\\', *p++ = 't';
@@ -934,7 +937,9 @@
*p++ = c;
}
assert(newsize - (p - PyUnicode_AS_UNICODE(v)) >= 1);
-*p++ = quote;
+while (*quote_postfix) {
+ *p++ = *quote_postfix++;
+}
*p = '\0';
if (PyUnicode_Resize(&v, (p - PyUnicode_AS_UNICODE(v)))
[issue1247] PEP 3137 patch (repr, names, parser)
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1247> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1247] PEP 3137 patch (repr, names, parser)
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1247> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1260] PEP 3137: Remove the buffer API from PyUnicode
New submission from Alexandre Vassalotti:
This patch removes the buffer API from PyUnicode, as specified by PEP
3137. All the unit tests passes. However, I believe there is a few
function calls to PyArg API that could become problematic if they are
given a PyUnicode object:
% egrep -R --include='*.c' 'PyArg.*Parse.*"[^;:]*[cwt].*"' py3k/
I haven't checked these function calls yet. So, it would probably be a
good idea to wait I do so before committing this patch.
--
files: unicode_rm_buf_api.patch
messages: 56336
nosy: alexandre.vassalotti
severity: normal
status: open
title: PEP 3137: Remove the buffer API from PyUnicode
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1260>
__Index: Modules/_sre.c
===
--- Modules/_sre.c (revision 58376)
+++ Modules/_sre.c (working copy)
@@ -1674,6 +1674,15 @@
void* ptr;
Py_buffer view;
+/* Unicode objects do not support the buffer API. So, get the data
+ directly instead. */
+if (PyUnicode_Check(string)) {
+ptr = (void *)PyUnicode_AS_DATA(string);
+*p_length = PyUnicode_GET_SIZE(string);
+*p_charsize = sizeof(Py_UNICODE);
+return ptr;
+}
+
/* get pointer to string buffer */
view.len = -1;
buffer = Py_Type(string)->tp_as_buffer;
Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58376)
+++ Objects/unicodeobject.c (working copy)
@@ -8113,19 +8113,6 @@
};
-static int
-unicode_buffer_getbuffer(PyUnicodeObject *self, Py_buffer *view, int flags)
-{
-
-if (flags & PyBUF_CHARACTER) {
-PyErr_SetString(PyExc_SystemError, "can't use str as char buffer");
-return -1;
-}
-return PyBuffer_FillInfo(view, (void *)self->str,
- PyUnicode_GET_DATA_SIZE(self), 1, flags);
-}
-
-
/* Helpers for PyUnicode_Format() */
static PyObject *
@@ -8819,11 +8806,6 @@
return NULL;
}
-static PyBufferProcs unicode_as_buffer = {
-(getbufferproc) unicode_buffer_getbuffer,
-NULL,
-};
-
static PyObject *
unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
@@ -8907,7 +8889,7 @@
(reprfunc) unicode_str, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
-&unicode_as_buffer, /* tp_as_buffer */
+0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */
unicode_doc, /* tp_doc */
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment:
I found a few problems in your patch. In PyCode_New() the type check
for the filename argument is incorrect:
--- Objects/codeobject.c(revision 58412)
+++ Objects/codeobject.c(working copy)
@@ -59,7 +59,7 @@
freevars == NULL || !PyTuple_Check(freevars) ||
cellvars == NULL || !PyTuple_Check(cellvars) ||
name == NULL || (!PyString_Check(name) && !PyUnicode_Check(name)) ||
- filename == NULL || !PyString_Check(filename) ||
+ filename == NULL || (!PyString_Check(name) &&
!PyUnicode_Check(name)) ||
lnotab == NULL || !PyString_Check(lnotab) ||
!PyObject_CheckReadBuffer(code)) {
PyErr_BadInternalCall();
@@ -260,6 +267,8 @@
ourcellvars = PyTuple_New(0);
if (ourcellvars == NULL)
goto cleanup;
+filename = PyUnicode_DecodeFSDefault(PyString_AS_STRING(filename),
+ 0, NULL);
The following is unnecessary and will cause a reference leak:
@@ -260,6 +267,8 @@
ourcellvars = PyTuple_New(0);
if (ourcellvars == NULL)
goto cleanup;
+filename = PyUnicode_DecodeFSDefault(PyString_AS_STRING(filename),
+ 0, NULL);
co = (PyObject *)PyCode_New(argcount, kwonlyargcount,
nlocals, stacksize, flags,
I think the interface of PyUnicode_DecodeFSDefault() could be improved
a bit. The function doesn't use the last argument 'errors', so why is
there? I am not sure if it is useful to keep second argument,
'length', either. So, I believe the function prototype should be
changed to:
PyObject *PyUnicode_Decode_FSDefault(const char *s);
Another thing that I am not sure about is whether it is correct to
consider ISO-8859-15 the same thing as Latin-1.
Overall, the patch looks good to me and doesn't cause any test to
fail. I attached an updated patch with the above issues fixed.
Thank you, Christian, for the patch. :)
--
nosy: +alexandre.vassalotti
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1272>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Changes by Alexandre Vassalotti: __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: Guido wrote: > Why copy the default encoding before mangling it? With a little > extra care you will only have to copy it once. Also, consider not > mangling at all, but assuming the encoding comes in a canonical form > -- several other functions assume that, e.g. PyUnicode_Decode() and > PyUnicode_AsEncodedString(). It is impossible guarantee that Py_FileSystemDefaultEncoding is normalized, since its value can be set using nl_langinfo(CODESET) during the bootstrapping process. PyUnicode_Decode() and other decoding/encoding functions use the codec module, which is not available during the early bootstrapping process, to normalize the encoding name. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1260] PEP 3137: Remove the buffer API from PyUnicode
Alexandre Vassalotti added the comment:
There was a problem with one of the call of PyArg_ParseTuple in the OS/2
version of listdir() in the posix module. I also clarified the error
message of the 't' format unit.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1260>
__Index: Python/getargs.c
===
--- Python/getargs.c (revision 58422)
+++ Python/getargs.c (working copy)
@@ -1250,7 +1250,7 @@
arg, msgbuf, bufsize);
if (pb == NULL || pb->bf_getbuffer == NULL)
return converterr(
-"string or read-only character buffer",
+"bytes or read-only character buffer",
arg, msgbuf, bufsize);
if ((*pb->bf_getbuffer)(arg, &view, PyBUF_CHARACTER) != 0)
Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58422)
+++ Objects/unicodeobject.c (working copy)
@@ -8113,19 +8113,6 @@
};
-static int
-unicode_buffer_getbuffer(PyUnicodeObject *self, Py_buffer *view, int flags)
-{
-
-if (flags & PyBUF_CHARACTER) {
-PyErr_SetString(PyExc_SystemError, "can't use str as char buffer");
-return -1;
-}
-return PyBuffer_FillInfo(view, (void *)self->str,
- PyUnicode_GET_DATA_SIZE(self), 1, flags);
-}
-
-
/* Helpers for PyUnicode_Format() */
static PyObject *
@@ -8819,11 +8806,6 @@
return NULL;
}
-static PyBufferProcs unicode_as_buffer = {
-(getbufferproc) unicode_buffer_getbuffer,
-NULL,
-};
-
static PyObject *
unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
@@ -8907,7 +8889,7 @@
(reprfunc) unicode_str, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
-&unicode_as_buffer, /* tp_as_buffer */
+0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */
unicode_doc, /* tp_doc */
Index: Modules/_sre.c
===
--- Modules/_sre.c (revision 58422)
+++ Modules/_sre.c (working copy)
@@ -1674,6 +1674,15 @@
void* ptr;
Py_buffer view;
+/* Unicode objects do not support the buffer API. So, get the data
+ directly instead. */
+if (PyUnicode_Check(string)) {
+ptr = (void *)PyUnicode_AS_DATA(string);
+*p_length = PyUnicode_GET_SIZE(string);
+*p_charsize = sizeof(Py_UNICODE);
+return ptr;
+}
+
/* get pointer to string buffer */
view.len = -1;
buffer = Py_Type(string)->tp_as_buffer;
Index: Modules/posixmodule.c
===
--- Modules/posixmodule.c (revision 58422)
+++ Modules/posixmodule.c (working copy)
@@ -2135,7 +2135,8 @@
FILEFINDBUF3 ep;
APIRET rc;
-if (!PyArg_ParseTuple(args, "t#:listdir", &name, &len))
+if (!PyArg_ParseTuple(args, "et#:listdir",
+ Py_FileSystemDefaultEncoding, &name, &len))
return NULL;
if (len >= MAX_PATH) {
PyErr_SetString(PyExc_ValueError, "path too long");
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment:
Guido wrote:
> I figured out the problem -- it came from marshalled old code objects.
> If you throw away all .pyc files the problem goes away. You can also
> get rid of the similar checks for the 'name' argument -- this should
> just be a PyUnicode too. A systematic approach to invalidating all the
> .pyc files is updating the magic number in import.c.
Done.
I had to remove a few another PyString instances in pyexpat.c and
_ctypes.c. So, here (hopefully) the final version of the patch.
The changes from the last version are:
- Correct a typo in of the comments in PyUnicode_DecodeFSDefault
- Specified in the API doc of PyUnicode_DecodeFSDefault that the
function take a null-terminated string.
- Bumped the magic number in import.c
- Fix PyCode_New calls in _ctypes and pyexpat module.
- Remove the PyString type check on 'filename' and 'name' in PyCode_New.
- Remove the unneeded string coercion code from PyCode_New.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1272>
__Index: Python/ceval.c
===
--- Python/ceval.c (revision 58422)
+++ Python/ceval.c (working copy)
@@ -767,7 +767,7 @@
lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL;
#endif
#if defined(Py_DEBUG) || defined(LLTRACE)
- filename = PyString_AsString(co->co_filename);
+ filename = PyUnicode_AsString(co->co_filename);
#endif
why = WHY_NOT;
Index: Python/traceback.c
===
--- Python/traceback.c (revision 58422)
+++ Python/traceback.c (working copy)
@@ -229,10 +229,10 @@
while (tb != NULL && err == 0) {
if (depth <= limit) {
err = tb_displayline(f,
- PyString_AsString(
+ PyUnicode_AsString(
tb->tb_frame->f_code->co_filename),
tb->tb_lineno,
- PyString_AsString(tb->tb_frame->f_code->co_name));
+ PyUnicode_AsString(tb->tb_frame->f_code->co_name));
}
depth--;
tb = tb->tb_next;
Index: Python/pythonrun.c
===
--- Python/pythonrun.c (revision 58422)
+++ Python/pythonrun.c (working copy)
@@ -867,7 +867,8 @@
return -1;
d = PyModule_GetDict(m);
if (PyDict_GetItemString(d, "__file__") == NULL) {
- PyObject *f = PyString_FromString(filename);
+ PyObject *f;
+ f = PyUnicode_DecodeFSDefault(filename);
if (f == NULL)
return -1;
if (PyDict_SetItemString(d, "__file__", f) < 0) {
Index: Python/import.c
===
--- Python/import.c (revision 58422)
+++ Python/import.c (working copy)
@@ -74,10 +74,11 @@
3040 (added signature annotations)
3050 (print becomes a function)
3060 (PEP 3115 metaclass syntax)
- 3070 (PEP 3109 raise changes)
+ 3070 (PEP 3109 raise changes)
+ 3080 (PEP 3137 make __file__ and __name__ unicode)
.
*/
-#define MAGIC (3070 | ((long)'\r'<<16) | ((long)'\n'<<24))
+#define MAGIC (3080 | ((long)'\r'<<16) | ((long)'\n'<<24))
/* Magic word as global; note that _PyImport_Init() can change the
value of this global to accommodate for alterations of how the
@@ -652,7 +653,7 @@
/* Remember the filename as the __file__ attribute */
v = NULL;
if (pathname != NULL) {
- v = PyString_FromString(pathname);
+ v = PyUnicode_DecodeFSDefault(pathname);
if (v == NULL)
PyErr_Clear();
}
@@ -983,7 +984,7 @@
PySys_WriteStderr("import %s # directory %s\n",
name, pathname);
d = PyModule_GetDict(m);
- file = PyString_FromString(pathname);
+ file = PyUnicode_DecodeFSDefault(pathname);
if (file == NULL)
goto error;
path = Py_BuildValue("[O]", file);
Index: Python/compile.c
===
--- Python/compile.c (revision 58422)
+++ Python/compile.c (working copy)
@@ -4001,7 +4001,7 @@
freevars = dict_keys_inorder(c->u->u_freevars, PyTuple_Size(cellvars));
if (!freevars)
goto error;
- filename = PyString_FromString(c->c_filename);
+ filename = PyUnicode_DecodeFSDefault(c->c_filename);
if (!filename)
goto error;
Index: Python/importdl.c
===
--- Python/importdl.c (revision 58422)
+++ Python/importdl.c (working copy)
@@ -62,7 +62,9 @@
return NULL;
}
/* Remember the filename as the __file__ attribute */
- if (PyModule_AddStringConstant(m, "__file__", pathname) < 0)
+ PyObject *path;
+ path = PyUnicode_DecodeFSDefault(pathname);
+ if (PyModule_AddObject(m, "__file__", path) < 0)
PyErr_Clear(); /
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment:
> Remove the PyString type check on 'filename' and 'name' in PyCode_New.
Oops. I removed one of the ! the checks by mistake.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1272>
__Index: Python/ceval.c
===
--- Python/ceval.c (revision 58454)
+++ Python/ceval.c (working copy)
@@ -767,7 +767,7 @@
lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL;
#endif
#if defined(Py_DEBUG) || defined(LLTRACE)
- filename = PyString_AsString(co->co_filename);
+ filename = PyUnicode_AsString(co->co_filename);
#endif
why = WHY_NOT;
Index: Python/traceback.c
===
--- Python/traceback.c (revision 58454)
+++ Python/traceback.c (working copy)
@@ -229,10 +229,10 @@
while (tb != NULL && err == 0) {
if (depth <= limit) {
err = tb_displayline(f,
- PyString_AsString(
+ PyUnicode_AsString(
tb->tb_frame->f_code->co_filename),
tb->tb_lineno,
- PyString_AsString(tb->tb_frame->f_code->co_name));
+ PyUnicode_AsString(tb->tb_frame->f_code->co_name));
}
depth--;
tb = tb->tb_next;
Index: Python/pythonrun.c
===
--- Python/pythonrun.c (revision 58454)
+++ Python/pythonrun.c (working copy)
@@ -867,7 +867,8 @@
return -1;
d = PyModule_GetDict(m);
if (PyDict_GetItemString(d, "__file__") == NULL) {
- PyObject *f = PyString_FromString(filename);
+ PyObject *f;
+ f = PyUnicode_DecodeFSDefault(filename);
if (f == NULL)
return -1;
if (PyDict_SetItemString(d, "__file__", f) < 0) {
Index: Python/import.c
===
--- Python/import.c (revision 58454)
+++ Python/import.c (working copy)
@@ -74,10 +74,11 @@
3040 (added signature annotations)
3050 (print becomes a function)
3060 (PEP 3115 metaclass syntax)
- 3070 (PEP 3109 raise changes)
+ 3070 (PEP 3109 raise changes)
+ 3080 (PEP 3137 make __file__ and __name__ unicode)
.
*/
-#define MAGIC (3070 | ((long)'\r'<<16) | ((long)'\n'<<24))
+#define MAGIC (3080 | ((long)'\r'<<16) | ((long)'\n'<<24))
/* Magic word as global; note that _PyImport_Init() can change the
value of this global to accommodate for alterations of how the
@@ -652,7 +653,7 @@
/* Remember the filename as the __file__ attribute */
v = NULL;
if (pathname != NULL) {
- v = PyString_FromString(pathname);
+ v = PyUnicode_DecodeFSDefault(pathname);
if (v == NULL)
PyErr_Clear();
}
@@ -983,7 +984,7 @@
PySys_WriteStderr("import %s # directory %s\n",
name, pathname);
d = PyModule_GetDict(m);
- file = PyString_FromString(pathname);
+ file = PyUnicode_DecodeFSDefault(pathname);
if (file == NULL)
goto error;
path = Py_BuildValue("[O]", file);
Index: Python/compile.c
===
--- Python/compile.c (revision 58454)
+++ Python/compile.c (working copy)
@@ -4001,7 +4001,7 @@
freevars = dict_keys_inorder(c->u->u_freevars, PyTuple_Size(cellvars));
if (!freevars)
goto error;
- filename = PyString_FromString(c->c_filename);
+ filename = PyUnicode_DecodeFSDefault(c->c_filename);
if (!filename)
goto error;
Index: Python/importdl.c
===
--- Python/importdl.c (revision 58454)
+++ Python/importdl.c (working copy)
@@ -62,7 +62,9 @@
return NULL;
}
/* Remember the filename as the __file__ attribute */
- if (PyModule_AddStringConstant(m, "__file__", pathname) < 0)
+ PyObject *path;
+ path = PyUnicode_DecodeFSDefault(pathname);
+ if (PyModule_AddObject(m, "__file__", path) < 0)
PyErr_Clear(); /* Not important enough to report */
if (_PyImport_FixupExtension(name, pathname) == NULL)
Index: Include/unicodeobject.h
===
--- Include/unicodeobject.h (revision 58454)
+++ Include/unicodeobject.h (working copy)
@@ -154,6 +154,7 @@
# define PyUnicode_DecodeASCII PyUnicodeUCS2_DecodeASCII
# define PyUnicode_DecodeCharmap PyUnicodeUCS2_DecodeCharmap
# define PyUnicode_DecodeLatin1 PyUnicodeUCS2_DecodeLatin1
+# define PyUnicode_DecodeFSDefault PyUnicodeUCS2_DecodeFSDefault
# define PyUnicode_DecodeRawUnicodeEscape PyUnicodeUCS2_DecodeRawUnicodeEscape
# define PyUnicode_DecodeUTF32 PyUnicodeUCS2_DecodeUTF32
# define PyUnicode_DecodeUTF32Stateful PyUnicodeUCS2_DecodeUTF32Stateful
@@ -245,6 +246,7 @@
# define PyUnicode_DecodeASCII PyUnicodeUCS4_DecodeASCII
# define PyUnicode_Deco
[issue1260] PEP 3137: Remove the buffer API from PyUnicode
Alexandre Vassalotti added the comment: Committed in r58455. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1260> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment:
Christian wrote:
> Alexandre's mangle loop doesn't do the same job as mine. Chars like _
> and - aren't removed from the encoding name and the if clauses don't
> catch for example UTF-8 or ISO-8859-1 only UTF8 or ISO8859-1.
That isn't true. My mangler does exactly the same thing as your
original one.
However, I forgot to add Py_CHARMASK to the calls of tolower() and
isalnum() which would cause problems on platforms with signed char.
> Also he has overseen a PyString_Check in the code repr function.
Fixed.
> We have to get the codecs up and Py_FileSystemEncoding before we can
> decode the filenames. :( I think that the problem needs much more
> attention and a proper fix.
Maybe adding a global variable, let's say _Py_Codecs_Ready, could be
used to notify PyUnicode_DecodeFSDefault that it can use
PyUnicode_Decode, instead of relying only on the built-in codecs. That
would be much simpler than changing boostrapping process.
Here yet another updated patch. The changes are the following:
- Add Py_CHARMASK to tolower() and isalnum() calls in
PyUnicode_DecodeFSDefault().
- Use PyUnicode_Check(), instead of PyString_Check(), in
code_repr().
- Update comments for co_filename and co_name in PyCodeObject
struct definition.
- Fix a PyString_AS_STRING(co->co_name) instance in compile.c
- Replace %S for %U in PyErr_Format() calls for substituting co_name.
One more thing, frozen.c needs to be updated. The module data contains
a code object with a PyString co_name. However, there is a bug in the
imp module (it doesn't detect the encoding from modelines, which cause
io.TextIOWrapper to crash) that prevents me from generating the data
for frozen.c.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1272>
__Index: Python/ceval.c
===
--- Python/ceval.c (revision 58455)
+++ Python/ceval.c (working copy)
@@ -767,7 +767,7 @@
lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL;
#endif
#if defined(Py_DEBUG) || defined(LLTRACE)
- filename = PyString_AsString(co->co_filename);
+ filename = PyUnicode_AsString(co->co_filename);
#endif
why = WHY_NOT;
@@ -2565,7 +2565,7 @@
if (argcount > co->co_argcount) {
if (!(co->co_flags & CO_VARARGS)) {
PyErr_Format(PyExc_TypeError,
-"%S() takes %s %d "
+"%U() takes %s %d "
"%spositional argument%s (%d given)",
co->co_name,
defcount ? "at most" : "exactly",
@@ -2599,7 +2599,7 @@
int j;
if (keyword == NULL || !PyUnicode_Check(keyword)) {
PyErr_Format(PyExc_TypeError,
-"%S() keywords must be strings",
+"%U() keywords must be strings",
co->co_name);
goto fail;
}
@@ -2622,7 +2622,7 @@
if (j >= co->co_argcount + co->co_kwonlyargcount) {
if (kwdict == NULL) {
PyErr_Format(PyExc_TypeError,
- "%S() got an unexpected "
+ "%U() got an unexpected "
"keyword argument '%S'",
co->co_name,
keyword);
@@ -2633,7 +2633,7 @@
else {
if (GETLOCAL(j) != NULL) {
PyErr_Format(PyExc_TypeError,
- "%S() got multiple "
+ "%U() got multiple "
"values for keyword "
"argument '%S'",
co->co_name,
@@ -2661,7 +2661,7 @@
continue;
}
PyErr_Format(PyExc_TypeError,
- "%S() needs keyword-only argument %S",
+ "%U() needs keyword-only argument %S",
co->co_name, name);
goto fail;
}
@@ -2671,7 +2671,7 @@
for (i = argcount; i < m; i++) {
if (GETLOCAL(i) == NULL) {
PyErr_Format(PyExc_TypeError,
- "%S() takes %s %d "
+ "%U() takes %s %d "
"%spositional argument%s "
"(%d given)",
co->co_name,
@@ -2699,7 +2699,7 @@
else {
if (argcount > 0 || kwcount > 0) {
PyErr_Format(PyExc_TypeError,
- "%S() takes no arguments (%d given)",
+ "%U() takes no arguments (%d given)",
co->co_name,
argcount + kwcount);
goto fail;
Index: Python/traceback.c
===
--- Python/traceback.c (revision 58455)
+++ Python/traceback.c (working copy)
@@ -229,10 +229,10 @@
while (tb != NULL && err == 0) {
if (depth <= limit) {
err = tb_displayline(f,
- PyString_AsString(
+ PyUnicode_AsString(
tb->tb_frame->f_code->co_filename),
tb->tb_lineno,
- PyString_AsStrin
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment:
> I have a question for Alexandre related to frozen.c -- why is there a
> mode line with an encoding involved in freezing hello.py?
For some reason which I don't know about, freeze.py tries to read all
the modules accessible from sys.path:
# collect all modules of the program
dir = os.path.dirname(scriptfile)
path[0] = dir
mf = modulefinder.ModuleFinder(path, debug, exclude, replace_paths)
The problem is the imp module, which modulefinder uses, does not
detect the encoding of the files from the mode-line. This causes
TextIOWrapper to crash when it tries to read modules using an encoding
other than ASCII or UTF-8. Here an example:
>>> import imp
>>> imp.find_module('heapq')[0].read()
Traceback (most recent call last):
...
UnicodeDecodeError: 'utf8' codec can't decode bytes in position
1428-1430: invalid data
I probably should open a seperate issue in the tracker for this,
though.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1272>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment:
I thought of another way to implement PyUnicode_DecodeFSDefault. If
Py_FileSystemDefaultEncoding is set, decode with the codecs module,
otherwise use UTF-8 + replace. This works because when
Py_FileSystemDefaultEncoding is initialized at the end of
Py_InitializeEx(), the codecs module is ready to be used. Here's what
it looks like:
PyObject*
PyUnicode_DecodeFSDefault(const char *s)
{
Py_ssize_t size = (Py_ssize_t)strlen(s);
/* During the early bootstrapping process, Py_FileSystemDefaultEncoding
can be undefined. If it is case, decode using UTF-8. The
following assumes
that Py_FileSystemDefaultEncoding is set to a built-in encoding
during the
bootstrapping process where the codecs aren't ready yet.
*/
if (Py_FileSystemDefaultEncoding) {
return PyUnicode_Decode(s, size,
Py_FileSystemDefaultEncoding,
"replace");
}
else {
return PyUnicode_DecodeUTF8(s, size, "replace");
}
}
It is not perfect, since the extra function calls in the codecs module
causes test_profile and test_doctest to fail. However, I think this is
much simpler that the previous versions.
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1272>
__Index: Python/ceval.c
===
--- Python/ceval.c (revision 58455)
+++ Python/ceval.c (working copy)
@@ -767,7 +767,7 @@
lltrace = PyDict_GetItemString(f->f_globals, "__lltrace__") != NULL;
#endif
#if defined(Py_DEBUG) || defined(LLTRACE)
- filename = PyString_AsString(co->co_filename);
+ filename = PyUnicode_AsString(co->co_filename);
#endif
why = WHY_NOT;
@@ -2565,7 +2565,7 @@
if (argcount > co->co_argcount) {
if (!(co->co_flags & CO_VARARGS)) {
PyErr_Format(PyExc_TypeError,
-"%S() takes %s %d "
+"%U() takes %s %d "
"%spositional argument%s (%d given)",
co->co_name,
defcount ? "at most" : "exactly",
@@ -2599,7 +2599,7 @@
int j;
if (keyword == NULL || !PyUnicode_Check(keyword)) {
PyErr_Format(PyExc_TypeError,
-"%S() keywords must be strings",
+"%U() keywords must be strings",
co->co_name);
goto fail;
}
@@ -2622,7 +2622,7 @@
if (j >= co->co_argcount + co->co_kwonlyargcount) {
if (kwdict == NULL) {
PyErr_Format(PyExc_TypeError,
- "%S() got an unexpected "
+ "%U() got an unexpected "
"keyword argument '%S'",
co->co_name,
keyword);
@@ -2633,7 +2633,7 @@
else {
if (GETLOCAL(j) != NULL) {
PyErr_Format(PyExc_TypeError,
- "%S() got multiple "
+ "%U() got multiple "
"values for keyword "
"argument '%S'",
co->co_name,
@@ -2661,7 +2661,7 @@
continue;
}
PyErr_Format(PyExc_TypeError,
- "%S() needs keyword-only argument %S",
+ "%U() needs keyword-only argument %S",
co->co_name, name);
goto fail;
}
@@ -2671,7 +2671,7 @@
for (i = argcount; i < m; i++) {
if (GETLOCAL(i) == NULL) {
PyErr_Format(PyExc_TypeError,
- "%S() takes %s %d "
+ "%U() takes %s %d "
"%spositional argument%s "
"(%d given)",
co->co_name,
@@ -2699,7 +2699,7 @@
else {
if (argcount > 0 || kwcount > 0) {
PyErr_Format(PyExc_TypeError,
- "%S() takes no arguments (%d given)",
+ "%U() takes no arguments (%d given)",
co->co_name,
argcount + kwcount);
goto fail;
Index: Python/traceback.c
===
--- Python/traceback.c (revision 58455)
+++ Python/traceback.c (working copy)
@@ -229,10 +229,10 @@
while (tb != NULL && err == 0) {
if (depth <= limit) {
err = tb_displayline(f,
- PyString_AsString(
+ PyUnicode_AsString(
tb->tb_frame->f_code->co_filename),
tb->tb_lineno,
- PyString_AsString(tb->tb_frame->f_code->co_name));
+ PyUnicode_AsString(tb->tb_frame->f_code->co_name));
}
depth--;
tb = tb->tb_next;
Index: Python/pythonrun.c
===
--- Python/pythonrun.c (revision 58455)
+++ Python/pythonrun.c (working copy)
@@ -867,7 +867,8 @@
return -1;
d = PyModule_GetDict(m);
if (PyDict_GetItemString(d, "__file__") == NULL) {
- PyObject *f = PyString_FromString(filename);
+ PyObject *f;
+ f = PyUnico
[issue1280] PEP 3137: Make PyString's indexing and iteration return integers
New submission from Alexandre Vassalotti:
Here a preliminary patch to make PyString return integers on indexing
and iteration. There is still quite a few XXX in the patch, that I would
like to fix. However, the good thing is all tests passes.
--
components: Interpreter Core
files: string_iter_ret_ints.patch
messages: 56442
nosy: alexandre.vassalotti
severity: normal
status: open
title: PEP 3137: Make PyString's indexing and iteration return integers
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1280>
__Index: Objects/stringobject.c
===
--- Objects/stringobject.c (revision 58458)
+++ Objects/stringobject.c (working copy)
@@ -1233,23 +1233,13 @@
static PyObject *
string_item(PyStringObject *a, register Py_ssize_t i)
{
- char pchar;
- PyObject *v;
+ if (i < 0)
+ i += Py_Size(a);
if (i < 0 || i >= Py_Size(a)) {
PyErr_SetString(PyExc_IndexError, "string index out of range");
return NULL;
}
- pchar = a->ob_sval[i];
- v = (PyObject *)characters[pchar & UCHAR_MAX];
- if (v == NULL)
- v = PyString_FromStringAndSize(&pchar, 1);
- else {
-#ifdef COUNT_ALLOCS
- one_strings++;
-#endif
- Py_INCREF(v);
- }
- return v;
+ return PyInt_FromLong((unsigned char)a->ob_sval[i]);
}
static PyObject*
@@ -5150,8 +5140,8 @@
assert(PyString_Check(seq));
if (it->it_index < PyString_GET_SIZE(seq)) {
- item = PyString_FromStringAndSize(
- PyString_AS_STRING(seq)+it->it_index, 1);
+ item = PyInt_FromLong(
+ (unsigned char)seq->ob_sval[it->it_index]);
if (item != NULL)
++it->it_index;
return item;
Index: Lib/modulefinder.py
===
--- Lib/modulefinder.py (revision 58458)
+++ Lib/modulefinder.py (working copy)
@@ -367,7 +367,7 @@
consts = co.co_consts
LOAD_LOAD_AND_IMPORT = LOAD_CONST + LOAD_CONST + IMPORT_NAME
while code:
-c = code[0]
+c = chr(code[0])
if c in STORE_OPS:
oparg, = unpack('t', c)[0] is not True:
+for c in b'\x01\x7f\xff\x0f\xf0':
+# XXX str8 constructor uses UTF-8 by default. So, to converting
+# XXX to int to a str8 of length-1 require this odd maneuver.
+x = str8(bytes(chr(255), 'latin-1'))
+if struct.unpack('>t', x)[0] is not True:
raise TestFailed('%c did not unpack as True' % c)
test_bool()
Index: Lib/test/string_tests.py
===
--- Lib/test/string_tests.py (revision 58458)
+++ Lib/test/string_tests.py (working copy)
@@ -558,6 +558,10 @@
a = self.type2test('DNSSEC')
b = self.type2test('')
for c in a:
+# Special case for the str8, since indexing returns a integer
+# XXX Maybe it would be a good idea to seperate str8's tests...
+if self.type2test == str8:
+c = chr(c)
b += c
hash(b)
self.assertEqual(hash(a), hash(b))
Index: Lib/test/test_bytes.py
===
--- Lib/test/test_bytes.py (revision 58458)
+++ Lib/test/test_bytes.py (working copy)
@@ -347,7 +347,7 @@
sample = str8("Hello world\n\x80\x81\xfe\xff")
buf = memoryview(sample)
b = bytes(buf)
-self.assertEqual(b, bytes(map(ord, sample)))
+self.assertEqual(b, bytes(sample))
def test_to_str(self):
sample = "Hello world\n\x80\x81\xfe\xff"
Index: Lib/dis.py
===
--- Lib/dis.py (revision 58458)
+++ Lib/dis.py (working copy)
@@ -117,8 +117,7 @@
extended_arg = 0
free = None
while i < n:
-c = code[i]
-op = ord(c)
+op = code[i]
if i in linestarts:
if i > 0:
print()
@@ -134,7 +133,7 @@
print(opname[op].ljust(20), end=' ')
i = i+1
if op >= HAVE_ARGUMENT:
-oparg = ord(code[i]) + ord(code[i+1])*256 + extended_arg
+oparg = code[i] + code[i+1]*256 + extended_arg
extended_arg = 0
i = i+2
if op == EXTENDED_ARG:
@@ -162,8 +161,7 @@
n = len(code)
i = 0
while i < n:
-c = code[i]
-op = ord(c)
+op = code[i]
if i == lasti: print('-->', end=' ')
else: print(' ', end=' ')
if i in labels: print('>>', end=' ')
@@ -172,7 +170,7 @@
print(opname[op].ljust(15), end=' ')
i = i+1
if op >= HAVE_ARGUMEN
[issue1278] imp.find_module() ignores -*- coding: Latin-1 -*-
Changes by Alexandre Vassalotti: -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1278> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1272] Decode __file__ and co_filename to unicode using fs default
Alexandre Vassalotti added the comment: I wrote in msg56419: > It is not perfect, since the extra function calls in the codecs module > causes test_profile and test_doctest to fail. How this was resolved? __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1272> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1302] Fixes for profile/cprofile
Alexandre Vassalotti added the comment:
I don't think it's possible to add shortcuts in PyUnicode_Decode for
UTF-16 and UTF-32 because the byte-order can be different depending of
the platform. So, these two need to pass through the codecs module.
I am sure if it's better, but I factored out the normalization routine
into its own function.
Added file: http://bugs.python.org/file8589/py3k_profile_fix-3.patch
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1302>
__Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58587)
+++ Objects/unicodeobject.c (working copy)
@@ -1049,29 +1049,55 @@
return NULL;
}
+static char *
+normalize(const char *enc)
+{
+register size_t i;
+size_t len = strlen(enc);
+char *p;
+
+p = PyMem_Malloc(len + 1);
+if (p == NULL)
+return NULL;
+for (i = 0; i < len; i++) {
+register char ch = enc[i];
+if (ch == ' ')
+ch = '-';
+else
+ch = tolower(Py_CHARMASK(ch));
+}
+p[i] = '\0';
+return p;
+}
+
PyObject *PyUnicode_Decode(const char *s,
- Py_ssize_t size,
- const char *encoding,
- const char *errors)
+ Py_ssize_t size,
+ const char *encoding,
+ const char *errors)
{
PyObject *buffer = NULL, *unicode;
Py_buffer info;
+char *enc;
if (encoding == NULL)
- encoding = PyUnicode_GetDefaultEncoding();
+encoding = PyUnicode_GetDefaultEncoding();
+enc = normalize(encoding);
+
/* Shortcuts for common default encodings */
-if (strcmp(encoding, "utf-8") == 0)
+if (strcmp(enc, "utf-8") == 0)
return PyUnicode_DecodeUTF8(s, size, errors);
-else if (strcmp(encoding, "latin-1") == 0)
+else if (strcmp(enc, "latin-1") == 0)
return PyUnicode_DecodeLatin1(s, size, errors);
#if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
-else if (strcmp(encoding, "mbcs") == 0)
+else if (strcmp(enc, "mbcs") == 0)
return PyUnicode_DecodeMBCS(s, size, errors);
#endif
-else if (strcmp(encoding, "ascii") == 0)
+else if (strcmp(enc, "ascii") == 0)
return PyUnicode_DecodeASCII(s, size, errors);
+PyMem_Free(enc);
+
/* Decode via the codec registry */
buffer = NULL;
if (PyBuffer_FillInfo(&info, (void *)s, size, 1, PyBUF_SIMPLE) < 0)
Index: Lib/test/regrtest.py
===
--- Lib/test/regrtest.py (revision 58587)
+++ Lib/test/regrtest.py (working copy)
@@ -1119,6 +1119,15 @@
if not os.path.supports_unicode_filenames:
self.expected.add('test_pep277')
+# doctest, profile and cProfile tests fail when the encoding
+# of the filesystem is not built-in, because of the extra calls
+# to the codecs module.
+builtin_enc = ("utf-8", "latin-1", "ascii", "mbcs")
+if sys.getfilesystemencoding().lower() not in builtin_enc:
+self.expected.add('test_profile')
+self.expected.add('test_cProfile')
+self.expected.add('test_doctest')
+
try:
from test import test_socket_ssl
except ImportError:
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1286] fileinput, StringIO, and cStringIO do not support the with protocol
Alexandre Vassalotti added the comment: Do you have a use-case for this? In Py3k, I don't think adding support for the 'with' statement to StringIO makes any sense, since the close() method does nothing. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1286> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1302] Fixes for profile/cprofile
Alexandre Vassalotti added the comment: > I still don't understand why you are using (sizeof lower) - 2 It is simply to avoid duplicating the constant (a.k.a. the Don't Repeat Yourself (DRY) rule). > and what &lower[(sizeof lower) - 2] returns. Is it the memory address > of lower[17]? It the address of lower[18] to be exact. (l < &lower[(sizeof lower) - 2]) is simply tricky notation to check the bound of the array. Personally, I used like to subtract pointer, ((lower - l + 1) < (sizeof lower)) to get the bound. But now, I find Guido's trick more cute (and less error-prone). :) __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1302> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1302] Fixes for profile/cprofile
Alexandre Vassalotti added the comment: > Personally, I used like to subtract pointer, ((lower - l + 1) < > (sizeof lower)) to get the bound. Doh. I got it backward. It's (l - lower + 1), not (lower - l + 1). > But now, I find Guido's trick more cute (and less error-prone). At least, I got that right. :) __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1302> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395] py3k: duplicated line endings when using read(1)
Alexandre Vassalotti added the comment: On 11/7/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 11/7/07, Christian Heimes <[EMAIL PROTECTED]> wrote: > > > > Christian Heimes added the comment: > > > > By the way what happened to the SoC project related to Python's new IO > > subsystem? IIRC somebody was working on a C optimization of the io lib. > > > > I think it was Alexandre Vassalotti. Is that right, Alexandre? Or am I > mixing you up? (If you ca, please respond to the bug.) I think so. My GSoC project was to merge the interface of cPickle/pickle and cStringIO/StringIO. I am still working on it, albeit slowly (my school homework is killing all my free time, right now). My work on StringIO and BytesIO is basically done; I just need to add newline translation and encoding support before it can be merged into the trunk. The cPickle/pickle merge is mostly done (i.e., all the current tests passes); I am right now in the bug-hunting phase. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1395> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395] py3k: duplicated line endings when using read(1)
Alexandre Vassalotti added the comment: On 11/7/07, Guido van Rossum wrote: > Cool. How hard do you think it would be to extend your work on > StringIO into a translation of TextIOWrapper into C? Well, StringIO and TextIOWrapper are quite different. The only part that I could reuse, from StringIO, would be the newline translation facilities. I think that a perfect translation to C of the current Python implementation of TextIOWrapper will be burdensome (due to things like function annotations) but not too hard to do. Nevertheless, that would be neat project for me. I could start to work it by mid-December, along with the other performance enhancements I have in mind. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1395> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13503] improved efficiency of bytearray pickling by using bytes type instead of str
Alexandre Vassalotti added the comment: Antoine, do we have unit tests for this code path? -- ___ Python tracker <http://bugs.python.org/issue13503> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13520] Patch to make pickle aware of __qualname__
Alexandre Vassalotti added the comment: This might not be the case anymore, but __module__ can sometime be None. There is some discussion about this in Issue 3657. We should define the expected behavior when this happens. Also, I don't think backward-compatibility of the protocol is a big issue if we use the getattr approach. I feel bumping the protocol version is only necessary if we introduce new opcodes or change their interpretation. -- ___ Python tracker <http://bugs.python.org/issue13520> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13505] Bytes objects pickled in 3.x with protocol <=2 are unpickled incorrectly in 2.x
Alexandre Vassalotti added the comment:
I think we are kind of stuck here. I might need to rely on some clever hack to
generate the desired str object in 2.7 without breaking the bytes support in
3.3 and without changing 2.7 itself.
One *dirty* trick I am thinking about would be to use something like
array.tostring() to construct the byte string.
from array import array
class bytes:
def __reduce__(self):
return (array.tostring, (array('B', self),))
Of course, this doesn't work because pickle doesn't method pickling. But, maybe
someone can figure out a way around this... I don't know.
Also, this is a bit annoying to fix since we changed the semantic meaning of
the STRING opcodes in 3.x---i.e., it now represents a unicode string instead of
a byte string.
--
___
Python tracker
<http://bugs.python.org/issue13505>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3635] pickle.dumps cannot save instance of dict-derived class that overrides __getattribute__
Alexandre Vassalotti added the comment: I don't think it is a bug. The posted code completely breaks the expected behavior of __getattribute__. With a such implementation, there is nothing we can do with this object as we cannot introspect it. Use the following if you really need this kind of behaviour: class E(dict): def __getattribute__(self,name): try: return self[name] except KeyError: return dict.__getattribute__(self, name) -- resolution: -> works for me status: open -> closed ___ Python tracker <http://bugs.python.org/issue3635> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13505] Bytes objects pickled in 3.x with protocol <=2 are unpickled incorrectly in 2.x
Alexandre Vassalotti added the comment: sbt, the bug is not that the encoding is inefficient. The problem is we cannot unpickle bytes streams from Python 3 using Python 2. -- ___ Python tracker <http://bugs.python.org/issue13505> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2775] Implement PEP 3108
Alexandre Vassalotti added the comment: Brett, issue 2919 had a patch that merges profile/cProfile for a while now but nobody test it yet. All I need it someone to download the patch, install it, test it on some random script and tell me if it works. I don't need more. -- ___ Python tracker <http://bugs.python.org/issue2775> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13505] Bytes objects pickled in 3.x with protocol <=2 are unpickled incorrectly in 2.x
Alexandre Vassalotti added the comment: Fixed. Thanks for the patch! -- assignee: -> alexandre.vassalotti resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue13505> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Alexandre Vassalotti added the comment: Antoine, I fixed these issues in the latest patch posted on Rietveld. Also, Skip added the buffer limit in Unladen Swallow (see msg112956). We just need to merge that. -- Added file: http://bugs.python.org/file18777/issue1694050_19001.diff.txt ___ Python tracker <http://bugs.python.org/issue9410> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3873] Unpickling is really slow
Alexandre Vassalotti added the comment: I get this error with the patch: python: /home/alex/src/python.org/py3k/Modules/_pickle.c:908: _Unpickler_ReadFromFile: Assertion `self->next_read_idx == 0' failed. Aborted -- ___ Python tracker <http://bugs.python.org/issue3873> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3873] Unpickling is really slow
Alexandre Vassalotti added the comment: Didn't Victor say that only one seek at the end is necessary per pickle? If this is the case, I don't think expensive seeks will be an issue. -- ___ Python tracker <http://bugs.python.org/issue3873> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9965] Loading malicious pickle may cause excessive memory usage
New submission from Alexandre Vassalotti : This was mentioned during the review of issue #9410 (http://codereview.appspot.com/1694050/diff/2001/3001#newcode347), however we forgot to fix this. The new array-based memo for the Unpickler class assumes incorrectly that memo indices are always contiguous. This is not the case. And due to this, the following pickle will cause Unpickler to use about 3GB of memory to store the memo array. ./python -c "import pickle; pickle.loads(b'\x80\x02]r\xff\xff\xff\x06.')" To fix this, we can add code to fall-back to a dictionary-based memo when the memo keys are not contiguous. -- components: Extension Modules messages: 117492 nosy: alexandre.vassalotti priority: critical severity: normal stage: needs patch status: open title: Loading malicious pickle may cause excessive memory usage type: security versions: Python 3.2 ___ Python tracker <http://bugs.python.org/issue9965> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9965] Loading malicious pickle may cause excessive memory usage
Changes by Alexandre Vassalotti : -- nosy: +pitrou ___ Python tracker <http://bugs.python.org/issue9965> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9965] Loading malicious pickle may cause excessive memory usage
Alexandre Vassalotti added the comment: I was going to say this method http://docs.python.org/dev/py3k/library/pickle.html#restricting-globals could be used to prevent this kind of attack on bytearray. But, I came up with this fun thing: pickle.loads(b'\x80\x03cbuiltins\nlist\ncbuiltins\nrange\nJ\xff\xff\xff\x03\x85R\x85R.') Sigh... you are right about pickle being insecure by design. The only solution is to use HMAC to check the integrity and the authenticity of incoming pickles. -- ___ Python tracker <http://bugs.python.org/issue9965> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9935] Faster pickling of instances
Alexandre Vassalotti added the comment: Sorry Antoine, I have been busy with school work lately. I like the general idea and I will try to look at your patch ASAP. -- ___ Python tracker <http://bugs.python.org/issue9935> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11299] Allow deepcopying and pickling paused generators
Alexandre Vassalotti added the comment: Although, I would really like to see support of pickling generators. It is not really possible in CPython. This is recurrent request. I even wrote a small article about it. http://peadrop.com/blog/2009/12/29/why-you-cannot-pickle-generators/ Looking how PyPy did it, I see portability problems with the fact it dumps the byte-code of the generator to disk. Python's documentation clearly states that the byte-code is an implementation details and can (and does) change between releases. Hence, this method is not really suitable for pickle which needs to remain compatible across releases. -- ___ Python tracker <http://bugs.python.org/issue11299> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11299] Allow deepcopying and pickling paused generators
Alexandre Vassalotti added the comment: The value of the instruction pointer depends on the byte-code. So it's not portable either. But, the bigger issue is the fact generator objects do not have names we can refer to, unlike top-level functions and classes which pickle supports. Similarly, we can't pickle lambdas and nested-functions for this exact reason. Personally, I haven't found a way around this. But, that doesn't mean there isn't one. If you find one, I will more than pleased to review it. -- ___ Python tracker <http://bugs.python.org/issue11299> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11564] pickle not 64-bit ready
Alexandre Vassalotti added the comment: We could resort to the text-based protocol which doesn't have these limitations with respect to object lengths (IIRC). Performance won't be amazing, but we won't have to modify the current pickle protocol. -- ___ Python tracker <http://bugs.python.org/issue11564> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2887] bsddb 4.6.4 needs to be ported to Python 3.0
New submission from Alexandre Vassalotti <[EMAIL PROTECTED]>:
The recent updates to bsddb (r63207, r63210 and r63218) needs to
forward-ported to the py3k branch.
At first glance, here is the things that needs to be done in the test suite:
- Change the import:
from test_all import ...
into a relative import:
from .test_all import ...
- Replace code incompatible with 3.0, such as changing
``dict.has_key(key)`` to ``key in dict``.
- Change str literals to bytes literals where appropriate.
- Optional: change code like ``type([])`` or ``type(())`` to
respectively ``list`` and ``tuple``.
- Change print statements into print() calls.
- Change ``x != None`` to ``x is not None``.
In the modules:
- Change PyInt__* to PyLong_*.
- Update the PyTypeObject declaration:
statichere PyTypeObject DB_Type = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"DB", /*tp_name*/
sizeof(DBObject), /*tp_basicsize*/
...
to:
static PyTypeObject DB_Type = {
PyVarObject_HEAD_INIT(NULL, 0)
"DB", /*tp_name*/
sizeof(DBObject), /*tp_basicsize*/
...
- Update module init declaration:
DL_EXPORT(void) init_bsddb(void)
{
...
to:
PyMODINIT_FUNC init_bsddb(void)
{
...
- Remove Py_TPFLAGS_HAVE_WEAKREFS.
- Change PyString_* calls to PyUnicode_* where appropriate.
There probably other things that I missed, but that should give you a
good start.
--
components: Extension Modules, Library (Lib), Tests
messages: 66918
nosy: alexandre.vassalotti, gregory.p.smith, jcea
priority: normal
severity: normal
status: open
title: bsddb 4.6.4 needs to be ported to Python 3.0
type: feature request
versions: Python 3.0
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2887>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2888] pprint produces different output in 2.6 and 3.0
New submission from Alexandre Vassalotti <[EMAIL PROTECTED]>: The indent argument produces different output in Python 2.6 and 3.0: Python 3.0a5+ (py3k:63349:63350M, May 16 2008, 00:37:17) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pprint >>> stuff = ['spam', 'eggs', 'lumberjack', 'knights', 'ni'] >>> stuff.insert(0, stuff[:]) >>> pprint.pprint(stuff, indent=4) [ ['spam', 'eggs', 'lumberjack', 'knights', 'ni'], 'spam', 'eggs', 'lumberjack', 'knights', 'ni'] Python 2.6a3+ (trunk:63323, May 15 2008, 16:09:01) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pprint >>> stuff = ['spam', 'eggs', 'lumberjack', 'knights', 'ni'] >>> stuff.insert(0, stuff[:]) >>> pprint.pprint(stuff, indent=4) [ [ 'spam', 'eggs', 'lumberjack', 'knights', 'ni'], 'spam', 'eggs', 'lumberjack', 'knights', 'ni'] -- components: Library (Lib) messages: 66919 nosy: alexandre.vassalotti priority: low severity: normal status: open title: pprint produces different output in 2.6 and 3.0 type: behavior versions: Python 2.6, Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2888> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2880] Rename repr to reprlib
Changes by Alexandre Vassalotti <[EMAIL PROTECTED]>: -- assignee: -> alexandre.vassalotti nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2880> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2880] Rename repr to reprlib
Changes by Alexandre Vassalotti <[EMAIL PROTECTED]>: -- resolution: -> fixed status: open -> closed __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2880> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2775] Implement PEP 3108
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Quentin Gallet-Gilles wrote: > I've found some places where configparser, copyreg, queue and > socketserver haven't been renamed. The attached patch > 'renaming_leftovers_2.6.patch' corrects this. Thanks! Applied in r63384. -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2775> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2874] Remove use of the stat module in the stdlib
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Brett wrote: > This will require moving over to using the named tuple features of > os.stat() along with adding methods to the object that the stat module > has as functions. How do you add new methods to a PyStructSequence? -- nosy: +alexandre.vassalotti __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2874> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2920] Patch to print symbolic value or errno in EnvironmentError.__str__()
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Overall, the patch looks good. I made a couple of fixes to make the changes more robust. -- nosy: +alexandre.vassalotti priority: -> normal type: -> feature request Added file: http://bugs.python.org/file10378/symbolic-errno.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2920> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2980] Pickle stream for unicode object may contain non-ASCII characters.
Changes by Alexandre Vassalotti <[EMAIL PROTECTED]>: -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2980> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Here is the fully reviewed and shiny C optimized pickle module. :-) Note, I am only posting the _pickle.c source code for now. I still need to work out a few details out. For example, there is a few methods that needs to be exposed in the public API -- i.e., Unpickler.find_class() and Pickler.save_reduce(). Also, instantiate() needs to reworked to match the one in pickle.py. Anyway, I am now pretty confident that I will have the time to finalize these details before the beta. Added file: http://bugs.python.org/file10501/_pickle.c ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Here is a diff against the previously posted _pickle.c module. The module is pretty much done now. I fixed find_class() to be a proper method and move the initialization code of Pickler/Unpickler into the tp_init slot to allow proper subclassing. The only things I didn't do is make save_reduce() public (since it would be useless) and fix instantiate() to be exactly the same as in pickle.py (since I have no idea how to cleanly inject a __class__ attribute into a pre-build empty class from C). With the beta postponed, I will probably add some optimizations to the module and update the stdlib documentation. -- keywords: +patch Added file: http://bugs.python.org/file10519/changeset-1.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2799] Remove PyUnicode_AsString(), rework PyUnicode_AsStringAndSize(), add PyUnicode_AsChar()
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I now think the proposed changes wouldn't be bad thing, after all. I have been bitten myself by the confusing naming of the Unicode API. So, there is definitely a potential for errors. The main problem with PyUnicode_AsString(), as Marc-André pointed out, is it doesn't follow the API signature of the rest of the Unicode API: char *PyUnicode_AsString(PyObject *unicode); PyObject *PyUnicode_AsUTF8String(PyObject *unicode); PyObject *PyUnicode_AsASCIIString(PyObject *unicode); On the other hand, I do like the simple API of PyUnicode_AsString. Also, I have to admit that the apparent similarity between the PyString and the PyUnicode API helped me to port my code to Py3K when I first started working on Python core. So, pragmatism might beat purity here. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2799> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2582] Unpickling of range objects fail in Py3k
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Weird. I wonder why it succeeds to pickle to range object at all. It seems the __reduce_ex__ method emits bogus value. >>> r.__reduce_ex__(2) (, (,), None, None, None) Anyway, that is easy to fix when you know the trick. So, here is the fix. -- assignee: -> alexandre.vassalotti keywords: +patch nosy: +alexandre.vassalotti Added file: http://bugs.python.org/file10530/fix_range_pickling.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2582> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2582] Unpickling of range objects fail in Py3k
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Oh, here is a slightly more efficient version. Added file: http://bugs.python.org/file10531/fix_range_pickling-2.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2582> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2582] Unpickling of range objects fail in Py3k
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: xrange() in Python 2.x is also affected by this bug. So, here is the fix. -- versions: +Python 2.5, Python 2.6 Added file: http://bugs.python.org/file10532/fix_xrange_pickling.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2582> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3008] Let bin() show floats
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment:
Kind of a cool hack. I am not sure how useful it is though. Personally,
I find ``struct.pack('d', f)`` slightly more educative (more useful
too), but that is just me.
And if you allow bin() to accept floats, will you make Python accept
binary float literals too?
--
nosy: +alexandre.vassalotti
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3008>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2523] binary buffered reading is quadratic
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I am going to go through your patch as soon as I get the time -- i.e., later today or tomorrow morning. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2523> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2523] binary buffered reading is quadratic
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I reviewed the patch and I found a few bugs -- i.e., peek() was replacing the buffer content, read() wasn't written in consideration of non-blocking streams, the removal of the None check in BufferedRandom.read() was wrong. Here's an updated patch. Added file: http://bugs.python.org/file10550/binaryio3.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2523> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3065] Fix pickling bug of collections.namedtuple
New submission from Alexandre Vassalotti <[EMAIL PROTECTED]>:
There is currently a pickling bug in the namedtuple factory:
>>> from collections import namedtuple
>>> MemoRecord = namedtuple("MemoRecord", "key, msg")
>>> m = MemoRecord(1,"hello")
>>> import pickle
>>> pickle.loads(pickle.dumps(m))
Traceback (most recent call last):
...
TypeError: __new__() takes exactly 3 positional arguments (2 given)
The bug is due to the fact that classes created by namedtuple don't
handle the __new__ arguments in the same fashion as tuple.__new__. The
fix is simply to define __getnewargs__.
--
assignee: rhettinger
components: Library (Lib)
files: fix_namedtuple_pickling.patch
keywords: patch, patch
messages: 67853
nosy: alexandre.vassalotti, rhettinger
priority: normal
severity: normal
status: open
title: Fix pickling bug of collections.namedtuple
type: behavior
versions: Python 2.6, Python 3.0
Added file: http://bugs.python.org/file10558/fix_namedtuple_pickling.patch
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3065>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Changes by Alexandre Vassalotti <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file10566/changeset-2.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Here is the full patch that adds the _pickle module. I would like to commit it as soon another developer tests it and (hopefully) reviews it. A documentation patch is coming as well. However since I don't want to block the release just for documentation patch, I will post it as a separate issue. Added file: http://bugs.python.org/file10567/add-cpickle-1.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2918] Merge StringIO/cStringIO in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Here's a preliminary patch that add the C optimization for StringIO. All tests are passing except two which depends on the StringIO.buffer attribute of the TextIOWrapper class. Honestly, I am not sure what is the correct way to fix this. I cannot simply "fake" the attribute by returning a BytesIO object, since the file position of buffer is undecidable. It seems to me that the only way to fix these failing tests would be to define a FakeIO class, in their test file, that would wrap ByteIO with TextIOWrapper, just like the old and inefficient StringIO. So, any idea on what would be the best thing to do? -- keywords: +patch Added file: http://bugs.python.org/file10568/add-stringio-1.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2918> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2523] binary buffered reading is quadratic
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Oh, that is simple to fix. You can round the value 2*avail to the nearest block by doing something like (2*avail) & ~(bksize-1) where bksize is a power of 2, or the less magic (2*avail//bksize) * bksize. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2523> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2582] Unpickling of range objects fail in Py3k
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Fixed in r64059 for Python 3.0 Fixed in r64056 and r64057 for Python 2.6 -- resolution: -> fixed status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2582> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2582] Unpickling of range objects fail in Py3k
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: > Fixed in r64056 and r64057 for Python 2.6 Oops, I meant r64057 and r64058. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2582> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1092962] Make Generators Pickle-able
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I think is a bad idea too. Unless I see patch that implements this feature cleanly, I will have to reject this feature request. -- nosy: +alexandre.vassalotti resolution: -> rejected status: open -> pending ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1092962> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1580] Use shorter float repr when possible
Changes by Alexandre Vassalotti <[EMAIL PROTECTED]>: -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1580> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2919] Merge profile/cProfile in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I will try to fix issue. I cannot promise that I will get it done before the beta though. -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2919> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2918] Merge StringIO/cStringIO in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Here's another patch fixes the failing tests. I have tried to support the buffer attribute using following hack: @property def buffer(self): # XXX Hack to support the buffer attribute. buf = codecs.getwriter(self.encoding)(BytesIO(), self.errors) value = self.getvalue() buf.write(value[:self.tell()]) pos = buf.stream.tell() buf.write(value[self.tell():]) buf.stream.seek(pos) return buf.stream but this doesn't work since some application might want to modify the buffer. So, I removed it. Another thing that bothered me were the bogus encoding and errors arguments. So, I also removed them. Added file: http://bugs.python.org/file10577/add-stringio-2.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2918> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2744] Fix test_cProfile
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I will try to fix this one with profile/cProfile merge. -- assignee: -> alexandre.vassalotti nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2744> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I updated the patch to use the new module framework. Added file: http://bugs.python.org/file10592/add-cpickle-2.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2918] Merge StringIO/cStringIO in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I updated the patch to use the new module framework. Added file: http://bugs.python.org/file10593/add-stringio-3.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2918> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2918] Merge StringIO/cStringIO in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Committed in r64154. -- resolution: -> accepted status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2918> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Committed in r64152. -- resolution: -> accepted status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2775] Implement PEP 3108
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Done with pickle and stringio. -- dependencies: -Merge StringIO/cStringIO in 3.0, merge pickle and cPickle in 3.0 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2775> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Okay, I fixed _pickle's integers unpickling on 64bit platforms. Here is the patch. Added file: http://bugs.python.org/file10601/fix_pickle_int64.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2917] merge pickle and cPickle in 3.0
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Restored _pickle in r64180. -- status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2917> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue616013] cPickle documentation incomplete
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Is this still desired? If so, I have fairly long list of differences that I could document. Or maybe, I could backport the _pickle module to Python 2.6, which doesn't have as many differences. -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue616013> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3350] multiprocessing adds built-in types to the global copyreg.dispatch_table
New submission from Alexandre Vassalotti <[EMAIL PROTECTED]>: The multiprocessing module modifies the global copyreg.dispatch_table to extend pickling to several built-in types (mostly callable types). In my humble opinion, this is unacceptable for a standard library module. Here is an example of a behaviour change made by multiprocessing: >>> import pickle >>> pickle.dumps(str.isalpha) Traceback (most recent call last): ... _pickle.PicklingError: Can't pickle : attribute lookup builtins.method_descriptor failed >>> import multiprocessing.util >>> pickle.dumps(str.isalpha) b'\x80\x03cbuiltins\ngetattr\nq\x00cbuiltins\nstr\nq\x01X\x07\x00\x00\x00isalphaq\x02\x86q\x03Rq\x04.' There was some discussion in issue 558238 about allowing methods to be pickled. However, no consensus was reached. In addition, there is a bug in the method pickling support added by multiprocessing in Python 3.0: Python 2.6b1+ (trunk:64899:64900, Jul 13 2008, 13:33:02) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import multiprocessing.util, pickle >>> class A(object): ... def foo(self): pass ... >>> pickle.dumps(A().foo, 2) '\x80\x02c__builtin__\ngetattr\nq\x00c__main__\nA\nq\x01)\x81q\x02}q\x03bU\x03fooq\x04\x86q\x05Rq\x06.' >>> pickle.dumps(A.foo, 2) '\x80\x02c__builtin__\ngetattr\nq\x00c__main__\nA\nq\x01U\x03fooq\x02\x86q\x03Rq\x04.' Python 3.0b1+ (py3k:64876M, Jul 11 2008, 12:20:51) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import multiprocessing.util, pickle >>> class A(object): ... def foo(self): pass ... >>> pickle.dumps(A().foo, 2) Traceback (most recent call last): ... pickle.PicklingError: Can't pickle : it's not found as builtins.method >>> pickle.dumps(A.foo, 2) Traceback (most recent call last): ... pickle.PicklingError: Can't pickle : it's not found as __main__.foo A better solution for the interprocess communication needs of multiprocessing would be to define custom subclass of Pickler with extended support for built-in types. If needed, I am willing to extend the _pickle module in Python 3.0 to support modification to its "dispatch table", like the undocumented dispatch_table attribute in pickle.Pickler. -- components: Library (Lib) messages: 69615 nosy: alexandre.vassalotti priority: normal severity: normal status: open title: multiprocessing adds built-in types to the global copyreg.dispatch_table type: behavior versions: Python 2.6, Python 3.0 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3350> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3119] pickle.py is limited by python's call stack
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: On some benchmark of my own, I get a good 2x slowdown when this patch is applied. The benchmark consists of loading ~1.4 millions objects (mostly dict, list, str and int) from a pickle string. It is basically a torture test for the inner function calls overhead. Anyway, the slowdown doesn't bother me much. I think a "stackless" pickle module would be well appreciated by many Python users. There is a few things that stops me from applying this patch. First, the patch will break subclasses of Pickler that relies on the save() method (Although the method is an implementation detail that shouldn't used). Therefore, any patches that modifies the API of save() is straight out for 2.x. And for Python 3.0, your patch should also remove recursion from the _pickle module (the transparent C accelerator for pickle). -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3119> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3274] Py_CLEAR(tmp) seg faults
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Committed the fix r64927. Thanks. -- nosy: +alexandre.vassalotti resolution: -> fixed status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3274> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3326] py3k shouldn't use -fno-strict-aliasing anymore
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: With gcc 4.2.3, I see a whole bunch of warnings: Objects/exceptions.c: In function ‘UnicodeDecodeError_init’: Objects/exceptions.c:1472: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/frameobject.c: In function ‘frame_setlineno’: Objects/frameobject.c:151: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicode_DecodeUTF7Stateful’: Objects/unicodeobject.c:1804: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:1815: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:1827: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeUTF8Stateful’: Objects/unicodeobject.c:2140: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2147: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeUTF32Stateful’: Objects/unicodeobject.c:2419: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2419: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2420: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2431: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeUTF16Stateful’: Objects/unicodeobject.c:2693: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2693: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2694: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2705: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeUnicodeEscape’: Objects/unicodeobject.c:2911: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2923: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:2962: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:3004: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:3018: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:3030: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeRawUnicodeEscape’: Objects/unicodeobject.c:3295: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:3327: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘_PyUnicode_DecodeUnicodeInternal’: Objects/unicodeobject.c:3502: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:3512: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeASCII’: Objects/unicodeobject.c:3875: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:3880: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_DecodeCharmap’: Objects/unicodeobject.c:4176: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:4226: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:4249: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:4276: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_Join’: Objects/unicodeobject.c:5724: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:5745: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c: In function ‘PyUnicodeUCS2_Format’: Objects/unicodeobject.c:8841: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:9158: warning: dereferencing type-punned pointer will break strict-aliasing rules Objects/unicodeobject.c:9223: warning: dereferencing type-punned pointer will break strict-aliasing rules Python/bltinmodule.c: In function ‘source_as_string’: Python/bltinmodule
[issue3153] sqlite leaks on error
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Fixed in r64930 (trunk) and r64931 (py3k). Thanks. -- nosy: +alexandre.vassalotti resolution: -> fixed status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3153> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3208] function annotation for builtin and C function
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Extension modules can use PyFunction_GetAnnotations() to access and modify the annotations dictionary. In addition, PyFunction_SetAnnotations() can be used to add annotations. I added some documentation for these functions in r64934. -- nosy: +alexandre.vassalotti resolution: -> works for me status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3208> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3385] cPickle to pickle conversion in py3k missing methods
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: The omission of the dispatch dictionary was sort of intentional. But now, I think it would be a good idea to add it to _pickle.Pickler. I will write a patch ASAP. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3385> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3385] cPickle to pickle conversion in py3k missing methods
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Just in case you are wondering why I haven't submitted a patch yet, I want to let you know that my home computer is currently broken. So, I won't be able to work on this until I get my computer fixed (which unfortunately could take a few weeks). ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3385> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2523] binary buffered reading is quadratic
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Antoine wrote: > Le lundi 21 juillet 2008 à 21:18 +, Martin v. Löwis a écrit : > > IIUC, a read of the full requested size would achieve exactly that: on a > > non-blocking stream (IIUC), a read will always return > > min(bytes_available, bytes_requested). > > Hmm, it seems logical indeed... Alexandre, do you have other information > on the subject? Martin is right. However, I don't how Python handle the case where bytes_available is zero (in C, an error value is returned and errno is set to EWOULDBLOCK). When I revised the patch I had a weak understanding of nonblocking I/O. I thought the "exponential" reads were for nonblocking I/O, but I see now that is non-sense. I am not sure, but I think Martin is also right about the second loop. The max() call should be changed back to "max(self.buffer_size, n))", like in the 2nd patch. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2523> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3431] multiprocessing uses Pickler.dispatch which isn't in 3.0 _pickle
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Duplicate of issue3385 -- resolution: -> duplicate status: open -> closed superseder: -> cPickle to pickle conversion in py3k missing methods ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3431> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2744] Fix test_cProfile
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Not anymore! :-) ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2744> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3492] Zlib compress/decompress functions returning bytearray
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Could you submit unified diff--i.e., with 'diff -u' or 'svn diff'? Also, could you add tests for this fix? -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3492> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3385] cPickle to pickle conversion in py3k missing methods
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I got a preliminary patch that adds the dispatch dictionary. However, the patch leaks and it doesn't expose the save_reduce() method (which is needed by ForkingPickler). -- keywords: +patch Added file: http://bugs.python.org/file11048/add_dispatch_check-0.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3385> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2389] Array pickling exposes internal memory representation of elements
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I don't see why this cannot be fixed easily. All we need to do is fix the __reduce__ method of array objects to emit a list--i.e. with array.tolist()--instead of a memory string. Since the reduce protocol is just a fancy way to store the constructor arguments, this won't break unpickling of array objects pickled by previous Python versions. And here is a patch against the trunk. -- keywords: +patch nosy: +alexandre.vassalotti Added file: http://bugs.python.org/file11096/fix_array_pickling.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2389> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3514] pickle segfault with infinite loop in __getattr__
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment:
This is a bug in the C implementation of pickle (i.e., the _pickle
module). I think you're right about the missing exception check. At
first glance, it looks like the missing else-if case for "setstate ==
NULL", in load_build(), is the cause of the problem:
static int
load_build(UnpicklerObject *self)
{
...
setstate = PyObject_GetAttrString(inst, "__setstate__");
if (setstate == NULL && PyErr_ExceptionMatches(PyExc_AttributeError)) {
PyErr_Clear();
}
/*---missing else-if case-
else if (setstate == NULL) {
return NULL;
}
--*/
else {
PyObject *result;
/* The explicit __setstate__ is responsible for everything. */
result = unpickler_call(self, setstate, state);
Py_DECREF(setstate);
if (result == NULL)
return -1;
Py_DECREF(result);
return 0;
}
...
--
nosy: +alexandre.vassalotti
priority: -> normal
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3514>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2389] Array pickling exposes internal memory representation of elements
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: The slowdown depends of the array type. The patch makes array unpickling a few orders of magnitude slower (i.e. between 4 and 15 times slower depending of the array type). In general, pickling is about as fast as with the binary representation (or faster!). Although since most 64-bit compilers uses the LP64 model, I think we could make a compromise and only pickle as a list arrays of long integers. This would fix the problem without any visible speed penalties. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2389> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2389] Array pickling exposes internal memory representation of elements
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: I'm all in for a standardized representation of array's pickles (with width and endianness preserved). However to happen, we will either need to change array's constructor to support at least the byte-order specification (like struct) or add built-in support for array in the pickle module (which could be done without modifying the pickle protocol). ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2389> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3514] pickle segfault with infinite loop in __getattr__
Alexandre Vassalotti <[EMAIL PROTECTED]> added the comment: Committed fix in r65689. Thanks! -- resolution: -> fixed status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3514> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
