Re: [Python-Dev] cStringIO.StringIO() buffer behavior
Guido van Rossum schrieb:
> Methinks that this was a fundamental limitation of cStringIO, not a
> bug. Certainly not something to be "fixed" in a bugfix release.
I'm sorry.
Martin v. Löwis schrieb:
>> See bugs #1548891 and #1730114.
>>
>> In the former, it was reported that cStringIO works differently from StringIO
>> when handling unicode strings; it used GetReadBuffer which returned the raw
>> internal UCS-2 or UCS-4 encoded string.
>>
>> I changed it to use GetCharBuffer, which converts to a string using the
>> default encoding first. This fix was also in 2.5.1.
>>
>> The latter bug now complains that this excludes things like array.array()s
>> from being used as an argument to cStringIO.StringIO(), which worked before
>> with GetReadBuffer.
>>
>> What's the preferred solution here?
>
> I think the 2.5.0 behavior to accept array.array should be restored (and
> a test case be added). What to do about Unicode strings, I don't know.
> I agree with Guido that they are officially not supported in cStringIO,
> so it would be best to reject them. OTOH, since 2.5.1 already supports
> them, another choice would be continue supporting them, in the same way
> as they are supported in 2.5.1. Either solution would special-case
> Unicode strings.
Okay, I propose the following patch:
Index: Modules/cStringIO.c
===
--- Modules/cStringIO.c (Revision 56763)
+++ Modules/cStringIO.c (Arbeitskopie)
@@ -673,12 +673,26 @@
char *buf;
Py_ssize_t size;
- if (PyObject_AsCharBuffer(s, (const char **)&buf, &size) != 0)
+ /* special-case Unicode objects: encode them in the default encoding */
+ if (PyUnicode_Check(s)) {
+s = PyUnicode_AsEncodedString(s, NULL, NULL);
+if (s == NULL)
return NULL;
+ } else {
+Py_INCREF(s);
+ }
+ if (PyObject_AsReadBuffer(s, (const char **)&buf, &size)) {
+PyErr_Format(PyExc_TypeError, "expected read buffer, %.200s found",
+ s->ob_type->tp_name);
+return NULL;
+ }
+
self = PyObject_New(Iobject, &Itype);
- if (!self) return NULL;
- Py_INCREF(s);
+ if (!self) {
+Py_DECREF(s);
+return NULL;
+ }
self->buf=buf;
self->string_size=size;
self->pbuf=s;
Georg
--
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cStringIO.StringIO() buffer behavior
On 8/6/07, Georg Brandl <[EMAIL PROTECTED]> wrote: > Guido van Rossum schrieb: > > Methinks that this was a fundamental limitation of cStringIO, not a > > bug. Certainly not something to be "fixed" in a bugfix release. > > I'm sorry. No problem. Somebody else should have flagged this, so it's our collective responsibility. > Okay, I propose the following patch: > > Index: Modules/cStringIO.c [...] My proposal is much more radical -- get rid of cStringIO altogether. (And also of StringIO.py.) There aren't that many places using it any more, and almost all of these are easily replaced with io.StringIO (or io.BytesIO!). There's already a fixer in 2to3 to do this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cStringIO.StringIO() buffer behavior
Oops, never mind. This was in the context of 2.5 and 2.6, but my reply was in the context of 3.0. Still, in the light of cStringIO disappearing, it would be good to keep cStringIO is stable as possible (probably restoring 2.5.0 behavior) so as to avoid breaking 3rd party code more than once. On 8/6/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 8/6/07, Georg Brandl <[EMAIL PROTECTED]> wrote: > > Guido van Rossum schrieb: > > > Methinks that this was a fundamental limitation of cStringIO, not a > > > bug. Certainly not something to be "fixed" in a bugfix release. > > > > I'm sorry. > > No problem. Somebody else should have flagged this, so it's our > collective responsibility. > > > Okay, I propose the following patch: > > > > Index: Modules/cStringIO.c > [...] > > My proposal is much more radical -- get rid of cStringIO altogether. > (And also of StringIO.py.) There aren't that many places using it any > more, and almost all of these are easily replaced with io.StringIO (or > io.BytesIO!). There's already a fixer in 2to3 to do this. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cStringIO.StringIO() buffer behavior
On 8/6/07, Georg Brandl <[EMAIL PROTECTED]> wrote:
> Okay, I propose the following patch:
> [...]
I think your patch is complicated for nothing. It would be much more
straightforward to use PyString_AsStringAndSize to encode the Unicode
string with the default encoding. I think it would be necessary to
port the fix to O_write and O_writelines.
-- Alexandre
Index: Modules/cStringIO.c
===
--- Modules/cStringIO.c (revision 56754)
+++ Modules/cStringIO.c (working copy)
@@ -665,8 +674,15 @@
char *buf;
Py_ssize_t size;
- if (PyObject_AsCharBuffer(s, (const char **)&buf, &size) != 0)
- return NULL;
+ /* Special case for unicode objects. */
+ if (PyUnicode_Check(s)) {
+ if (PyString_AsStringAndSize(s, &buf, &size) == -1)
+ return NULL;
+ }
+ else {
+ if (PyObject_AsReadBuffer(s, (const void **)&buf, &size) == -1)
+ return NULL;
+ }
self = PyObject_New(Iobject, &Itype);
if (!self) return NULL;
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pleaswe help with the countdown to zero failing tests in the struni branch!
We're down to 11 failing test in the struni branch. I'd like to get this down to zero ASAP so that we can retire the old p3yk (yes, with typo!) branch and rename py3k-struni to py3k. Please help! Here's the list of failing tests: test_ctypes Recently one test started failing again, after Martin changed PyUnicode_FromStringAndSize() to use UTF8 instead of Latin1. test_email test_email_codecs test_email_renamed Can someone contact the email-sig and ask for help with these? test_minidom Recently started failing again; probably shallow. test_sqlite Virgin territory, probably best done by whoever wrote the code or at least someone with time to spare. test_tarfile Virgin territory again (but different owner :-). test_urllib2_localnet test_urllib2net I think Jeremy Hylton may be close to fixing these, he's done a lot of work on urllib and httplib. test_xml_etree_c Virgin territory again. There are also a few tests that only fail on CYGWIN or OSX; I won't bother listing these. If you want to help, please refer to this wiki page: http://wiki.python.org/moin/Py3kStrUniTests There are also other tasks; see http://wiki.python.org/moin/Py3kToDo -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
