[ python-Bugs-1117302 ] sgmllib.SGMLParser
Bugs item #1117302, was opened at 2005-02-06 14:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117302&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Paul Birnie (pbirnie) Assigned to: Nobody/Anonymous (nobody) Summary: sgmllib.SGMLParser Initial Comment: sgmllib.SGMLParser calls start tag and end_methods correctly until it encounters One Two Three the seems to cause its parsing to become confused and I conly get call backs for tag a twice (link 1 and 3) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117302&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1112856 ] patch 1079734 broke cgi.FieldStorage w/ multipart post req.
Bugs item #1112856, was opened at 2005-01-31 01:58
Message generated for change (Comment added) made by irmen
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112856&group_id=5470
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Irmen de Jong (irmen)
Assigned to: Nobody/Anonymous (nobody)
Summary: patch 1079734 broke cgi.FieldStorage w/ multipart post req.
Initial Comment:
Patch 1079734 "Make cgi.py use email instead of rfc822
or mimetools" seems to have broken the cgi.FieldStorage
in cases where the request is a multipart post (for
instance, when a file upload form field is used).
See the attached test program.
With cgi.py revision <1.83 (python 2.4 for instance) I
get the expected results;
374
FieldStorage(None, None, [FieldStorage('param1', None,
'Value of param1'), FieldStorage('param2', None, 'Value
of param2'), FieldStorage('file', '', ''),
FieldStorage(None, None, '')])
but with cgi.py rev 1.83 (current) I get this:
374
FieldStorage(None, None, [FieldStorage('param1', None,
'')])
Another thing that I observed (which isn't reproduced
by this test program) is that cgi.FieldStorage.__init__
never completes when the fp is a socket-file (and the
request is again a multipart post). It worked fine with
the old cgi.py.
--
>Comment By: Irmen de Jong (irmen)
Date: 2005-02-06 23:58
Message:
Logged In: YES
user_id=129426
Yes, I'll try to make a test case for that within the next
few days.
--
Comment By: Josh Hoyt (joshhoyt)
Date: 2005-02-05 00:02
Message:
Logged In: YES
user_id=693077
Irmen, can you try to create a test case where the
cgi.FieldStorage never completes, so I can make sure that
any fix I come up with resolves it?
I will try to put together an implementation where the email
parser parses the whole multipart message.
--
Comment By: Irmen de Jong (irmen)
Date: 2005-02-04 23:06
Message:
Logged In: YES
user_id=129426
Johannes: while your patch makes my cgibug.py test case run
fine, it has 2 problems:
1- it runs much slower than the python2.4 code (probably
because of the reading back thing Josh is talking about);
2- it still doesn't fix the second problem that I observed:
cgi.FieldStorage never completes when fp is a socket. I
don't have a separate test case for this yet, sorry.
So Josh: perhaps your idea doesn't have these 2 problems?
--
Comment By: Josh Hoyt (joshhoyt)
Date: 2005-02-04 16:45
Message:
Logged In: YES
user_id=693077
Johannes, your patch looks fine to me. It would be nice if
we didn't have to keep reading back each part from the
parsed message, though.
I had an idea for another approach. Use email to parse the
MIME message fully, then convert it to FieldStorage fields.
Parsing could go something like:
== CODE ==
from email.FeedParser import FeedParser
parser = FeedParser()
# Create bogus content-type header...
parser.feed('Content-type: %s ; boundary=%s \r\n\r\n' %
(self.type, self.innerboundary))
parser.feed(self.fp.read())
message = parser.close()
# Then take parsed message and convert to FieldStorage fields
== END CODE ==
This lets the email parser handle all of the complexities of
MIME, but it does mean that we have to accurately re-create
all of the necessary headers. I can cook up a full patch if
anyone thinks this would fly.
--
Comment By: Johannes Gijsbers (jlgijsbers)
Date: 2005-02-04 11:15
Message:
Logged In: YES
user_id=469548
Here's a patch. We're interested in two things in the
patched loop:
* the rest of the multipart/form-data, including the headers
for the current part, for consumption by HeaderParser (this
is the tail variable)
* the rest of the multipart/form-data without the headers
for the current part, for consumption by FieldStorage (this
is the message.get_payload() call)
Josh, Irmen, do you see any problems with this patch?
(BTW, this fix should be ported to the parse_multipart
function as well, when I check it in (and I should make
cgibug.py into a test))
--
Comment By: Josh Hoyt (joshhoyt)
Date: 2005-02-04 08:21
Message:
Logged In: YES
user_id=693077
The email.HeaderParser module that is now used for parsing
headers consumes the rest of the body of the message when it
is used. If there was some way of telling the parser to stop
reading when it got to the end of the headers, I think this
problem would be fixed. Unfortunately, there is no external
interface to the email module's parser that lets us do this.
I hope we can come up with a reasonable solution that does
not involve reverting to using deprecated modules.
Thank
[ python-Bugs-1117556 ] SimpleHTTPServer and mimetypes: almost together
Bugs item #1117556, was opened at 2005-02-06 18:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117556&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Matthew L Daniel (mdaniel) Assigned to: Nobody/Anonymous (nobody) Summary: SimpleHTTPServer and mimetypes: almost together Initial Comment: SimpleHTTPServer.py from python2.4 (and python2.3, so this is an old bug that hasn't bothered anyone before now), uses mimetypes for mime-type detection. The issue is that it only uses 50% of mimetypes, in two different ways. I argue that SimpleHTTPServer should not be copying mimetypes type-map into its own variable instead of leveraging the functionality found in mimetypes.guess_type (which guesses without regard to case, btw). If, however, you guys stick with your own extension_map, this bug is really about calling ``if not mimetypes.inited: mimetypes.init()'' before doing any such copying. This allows mimetypes to find mime.types on the local host and populate the types_map with lots of meaningful mime types. If it doesn't find any mime.types, no harm done. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117556&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1069160 ] PyThreadState_SetAsyncExc segfault
Bugs item #1069160, was opened at 2004-11-18 20:48 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1069160&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 6 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: PyThreadState_SetAsyncExc segfault Initial Comment: PyThreadState_SetAsyncExc() crawls over the list of tstates without holding a mutex. Python implementation code has tried to get away with this kind of thing before (& more than once : and segfaults are always eventually reported. A common cause is that PyThreadState_DeleteCurrent() is typically called *without* the GIL held, so any thread can try to delete its own tstate *while* PyThreadState_SetAsyncExc() is trying to muck with that tstate. That the GIL is held by PyThreadState_SetAsyncExc's caller is no protection. Instead the code has to serialize against tstate chain mutations by acquiring head_mutex for the duration. Of course this is rare and can be virtually impossible to reproduce, but that makes the bug worse, not "better". -- >Comment By: Guido van Rossum (gvanrossum) Date: 2005-02-06 19:44 Message: Logged In: YES user_id=6380 But it was Just's idea, so assigning to him. Q for Just: would applying the patch cause any problems in your code that's using this feature? (Q. for Tim: hoes does the head_mutex in this file relate to the GIL? The ZAP() call would seem to require that the GIL is already head when this fn. is called, because it could invoke arbitrary Python __del__ code. Is there any other call that could acquire these locks in the opposite order, causing deadlocks?) -- Comment By: Tim Peters (tim_one) Date: 2005-02-03 11:41 Message: Logged In: YES user_id=31435 Yup, that patch is all I had in mind. It's a shame that nobody cares enough about this function to test it (which isn't a dig at you, Armin -- I don't care enough either <0.5 wink>). BTW, I have no idea why this function warns about return values greater than 1. It's possible that the original author realized the tstate chain could mutate while the loop was running, and that's where "greater than 1" came from. If so, locking head_mutex should make that impossible ... OK, some quality time with CVS shows that Guido checked this function in, so assigning the report to him. -- Comment By: Armin Rigo (arigo) Date: 2005-02-02 06:19 Message: Logged In: YES user_id=4771 I guess that the patch you have in mind is just to protect the loop with HEAD_LOCK/HEAD_UNLOCK. I cannot test this patch (attached), though, as I don't have any code that uses this function. I can only tell that it also looks like a reasonable thing to do to me. -- Comment By: Tim Peters (tim_one) Date: 2004-11-19 10:32 Message: Logged In: YES user_id=31435 Changed Category to Threads. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1069160&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1069160 ] PyThreadState_SetAsyncExc segfault
Bugs item #1069160, was opened at 2004-11-18 20:48 Message generated for change (Settings changed) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1069160&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 6 Submitted By: Tim Peters (tim_one) >Assigned to: Just van Rossum (jvr) Summary: PyThreadState_SetAsyncExc segfault Initial Comment: PyThreadState_SetAsyncExc() crawls over the list of tstates without holding a mutex. Python implementation code has tried to get away with this kind of thing before (& more than once : and segfaults are always eventually reported. A common cause is that PyThreadState_DeleteCurrent() is typically called *without* the GIL held, so any thread can try to delete its own tstate *while* PyThreadState_SetAsyncExc() is trying to muck with that tstate. That the GIL is held by PyThreadState_SetAsyncExc's caller is no protection. Instead the code has to serialize against tstate chain mutations by acquiring head_mutex for the duration. Of course this is rare and can be virtually impossible to reproduce, but that makes the bug worse, not "better". -- Comment By: Guido van Rossum (gvanrossum) Date: 2005-02-06 19:44 Message: Logged In: YES user_id=6380 But it was Just's idea, so assigning to him. Q for Just: would applying the patch cause any problems in your code that's using this feature? (Q. for Tim: hoes does the head_mutex in this file relate to the GIL? The ZAP() call would seem to require that the GIL is already head when this fn. is called, because it could invoke arbitrary Python __del__ code. Is there any other call that could acquire these locks in the opposite order, causing deadlocks?) -- Comment By: Tim Peters (tim_one) Date: 2005-02-03 11:41 Message: Logged In: YES user_id=31435 Yup, that patch is all I had in mind. It's a shame that nobody cares enough about this function to test it (which isn't a dig at you, Armin -- I don't care enough either <0.5 wink>). BTW, I have no idea why this function warns about return values greater than 1. It's possible that the original author realized the tstate chain could mutate while the loop was running, and that's where "greater than 1" came from. If so, locking head_mutex should make that impossible ... OK, some quality time with CVS shows that Guido checked this function in, so assigning the report to him. -- Comment By: Armin Rigo (arigo) Date: 2005-02-02 06:19 Message: Logged In: YES user_id=4771 I guess that the patch you have in mind is just to protect the loop with HEAD_LOCK/HEAD_UNLOCK. I cannot test this patch (attached), though, as I don't have any code that uses this function. I can only tell that it also looks like a reasonable thing to do to me. -- Comment By: Tim Peters (tim_one) Date: 2004-11-19 10:32 Message: Logged In: YES user_id=31435 Changed Category to Threads. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1069160&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1117601 ] os.path.exists returns false negatives in MAC environments.
Bugs item #1117601, was opened at 2005-02-07 00:57 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117601&group_id=5470 Category: Python Library Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Stephen Bennett (sbennett) Assigned to: Nobody/Anonymous (nobody) Summary: os.path.exists returns false negatives in MAC environments. Initial Comment: In Mandatory Access Control environments (such as SELinux), it's quite possible for stat to fail with permission denied. In this case, os.path.exists will return False incorrectly. The simple(ish) fix is to check for an access denied error (which would indicate present, but not readable) when using stat to check for existence of files. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117601&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1117048 ] Dictionary Evaluation Issue
Bugs item #1117048, was opened at 2005-02-05 17:50 Message generated for change (Settings changed) made by walterbrunswick You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None >Priority: 2 Submitted By: WalterBrunswick (walterbrunswick) Assigned to: Nobody/Anonymous (nobody) Summary: Dictionary Evaluation Issue Initial Comment: Python Version: Python 2.4 OS Version: Microsoft Windows XP [Version 5.1.2600] There may or may not be (according to some people on #python on irc.freenode.net) a fault, if you will, in the way that Python evaluates dictionaries (in this case, globals()). apparently printing of dicts are optimized not to produce the entire output string before printing it This may be an advantage when it comes to processing speed and memory management, but when it comes to catching errors, this could very well be a problem. (I am waltz.) ... you think it should print a dict with an error in it? ha! caught ya there! well, if there's an error, it should break. and it did. It shouldn't print AT ALL. why not? Because of the ERROR! ... I'm saying it should raise an abstract exception (something like DictError) BEFORE printing, as to allow the programmer to handle the error. Some people don't agree with my argument, but I'll let you (the core developers) decide what's best. I just thought I should mention this issue, to help improve Python to the max! -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1117048 ] Dictionary Evaluation Issue
Bugs item #1117048, was opened at 2005-02-05 17:50 Message generated for change (Settings changed) made by walterbrunswick You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None >Priority: 1 Submitted By: WalterBrunswick (walterbrunswick) Assigned to: Nobody/Anonymous (nobody) Summary: Dictionary Evaluation Issue Initial Comment: Python Version: Python 2.4 OS Version: Microsoft Windows XP [Version 5.1.2600] There may or may not be (according to some people on #python on irc.freenode.net) a fault, if you will, in the way that Python evaluates dictionaries (in this case, globals()). apparently printing of dicts are optimized not to produce the entire output string before printing it This may be an advantage when it comes to processing speed and memory management, but when it comes to catching errors, this could very well be a problem. (I am waltz.) ... you think it should print a dict with an error in it? ha! caught ya there! well, if there's an error, it should break. and it did. It shouldn't print AT ALL. why not? Because of the ERROR! ... I'm saying it should raise an abstract exception (something like DictError) BEFORE printing, as to allow the programmer to handle the error. Some people don't agree with my argument, but I'll let you (the core developers) decide what's best. I just thought I should mention this issue, to help improve Python to the max! -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1069160 ] PyThreadState_SetAsyncExc segfault
Bugs item #1069160, was opened at 2004-11-18 20:48
Message generated for change (Comment added) made by tim_one
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1069160&group_id=5470
Category: Threads
Group: Python 2.4
Status: Open
Resolution: None
Priority: 6
Submitted By: Tim Peters (tim_one)
Assigned to: Just van Rossum (jvr)
Summary: PyThreadState_SetAsyncExc segfault
Initial Comment:
PyThreadState_SetAsyncExc() crawls over the list of
tstates without holding a mutex. Python implementation
code has tried to get away with this kind of thing before
(& more than once : and segfaults are always
eventually reported.
A common cause is that PyThreadState_DeleteCurrent()
is typically called *without* the GIL held, so any thread
can try to delete its own tstate *while*
PyThreadState_SetAsyncExc() is trying to muck with
that tstate. That the GIL is held by
PyThreadState_SetAsyncExc's caller is no protection.
Instead the code has to serialize against tstate chain
mutations by acquiring head_mutex for the duration.
Of course this is rare and can be virtually impossible to
reproduce, but that makes the bug worse, not "better".
--
>Comment By: Tim Peters (tim_one)
Date: 2005-02-06 20:51
Message:
Logged In: YES
user_id=31435
It's already documented that PyThreadState_SetAsyncExc()
must be called with the GIL held. head_mutex is only used to
protect traversal/mutation of the threadstate chain, so is
only held internally by functions that create, or destroy,
thread states or interpreter states. It's certainly possible to
get a deadlock then after the patch, not due to the GIL, but
due to head_mutex alone. I doubt that any __del__ method
would create/destroy thread/interpreter states on its own,
the bigger danger is that once a __del__ method is invoked,
other threads can run too, and they may do anything
(including calling PyThreadState_SetAsyncExc() again! that
seem the most likely way to deadlock).
It would be good to have a clue about "I have no idea why
this function warns about return values greater than 1". If in
fact it should be impossible to have a count greater than 1
given that concurrent mutations to the tstate chain are
locked out after the patch, then the patch could be rewritten
to release head_mutex when it found p->thread_id == id the
first time, before the ZAP. After the ZAP, the function would
just return:
if (p->thread_id == id) {
PyObject *temp = p->async_exc;
Py_XINCREF(exc);
p->async_exc = exc;
HEAD_UNLOCK();
Py_XDECREF(temp);
return 1;
}
Then deadlock couldn't occur.
I should note that PyInterpreterState_Clear() is prone to the
same kind of deadlocks on head_mutex (it calls
PyThreadState_Clear() with head_mutex held, and the latter
does a lot of ZAPping).
--
Comment By: Guido van Rossum (gvanrossum)
Date: 2005-02-06 19:44
Message:
Logged In: YES
user_id=6380
But it was Just's idea, so assigning to him. Q for Just:
would applying the patch cause any problems in your code
that's using this feature?
(Q. for Tim: hoes does the head_mutex in this file relate to
the GIL? The ZAP() call would seem to require that the GIL
is already head when this fn. is called, because it could
invoke arbitrary Python __del__ code. Is there any other
call that could acquire these locks in the opposite order,
causing deadlocks?)
--
Comment By: Tim Peters (tim_one)
Date: 2005-02-03 11:41
Message:
Logged In: YES
user_id=31435
Yup, that patch is all I had in mind. It's a shame that nobody
cares enough about this function to test it (which isn't a dig
at you, Armin -- I don't care enough either <0.5 wink>).
BTW, I have no idea why this function warns about return
values greater than 1. It's possible that the original author
realized the tstate chain could mutate while the loop was
running, and that's where "greater than 1" came from. If so,
locking head_mutex should make that impossible ... OK, some
quality time with CVS shows that Guido checked this function
in, so assigning the report to him.
--
Comment By: Armin Rigo (arigo)
Date: 2005-02-02 06:19
Message:
Logged In: YES
user_id=4771
I guess that the patch you have in mind is just to protect the loop with
HEAD_LOCK/HEAD_UNLOCK. I cannot test this patch (attached), though, as I don't
have any code that uses this function. I can only tell that it also looks like
a reasonable thing to do to me.
--
Comment By: Tim Peters (tim_one)
Date: 2004-11-19 10:32
Message:
Logged In: YES
user_id=31435
Changed Category to Threads.
-
[ python-Bugs-1117670 ] profiler: Bad return and Bad call errors with exceptions
Bugs item #1117670, was opened at 2005-02-06 21:50
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117670&group_id=5470
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthew Mueller (donut)
Assigned to: Nobody/Anonymous (nobody)
Summary: profiler: Bad return and Bad call errors with exceptions
Initial Comment:
I ran into a weird error when trying to profile a test
script of mine:
AssertionError: ('Bad call', ('', 0, 'encode'))
I managed to whittle it down to some minimal test
cases, which are attached (although the errors they
generate are slightly different.)
$ python-cvs -m profile profile_badcall.py
Traceback (most recent call last):
[snipped ...]
File
"/home/donut/usr64/python/lib/python2.5/profile.py",
line 444, in runctx
exec cmd in globals, locals
File "", line 1, in ?
File "profile_badcall.py", line 10, in ?
os.path.join("C",'b')
File
"/home/donut/usr64/python/lib/python2.5/posixpath.py",
line 56, in join
def join(a, *p):
File
"/home/donut/usr64/python/lib/python2.5/profile.py",
line 228, in trace_dispatch_i
if self.dispatch[event](self, frame, t):
File
"/home/donut/usr64/python/lib/python2.5/profile.py",
line 285, in trace_dispatch_call
assert (self.cur is None or AssertionError: ('Bad call',
('profile_badcall.py', 2,
'trier'))
$ python-cvs -m profile profile_badreturn.py
Traceback (most recent call last):
[snipped...]
File
"/home/donut/usr64/python/lib/python2.5/profile.py",
line 444, in runctx
exec cmd in globals, locals
File "", line 1, in ?
File
"/home/donut/usr64/python/lib/python2.5/profile.py",
line 228, in trace_dispatch_i
if self.dispatch[event](self, frame, t):
File
"/home/donut/usr64/python/lib/python2.5/profile.py",
line 312, in trace_dispatch_return
assert frame is self.cur[-2].f_back, ("Bad return",
self.cur[-3])
AssertionError: ('Bad return', ('profile_badreturn.py',
1, 'trier'))
The errors occur in python CVS as of 20050206 and
python 2.4, but not in python 2.3.4.
OS: debian sid (3.1)
Arch: amd64
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117670&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1103023 ] raw_input problem with readline and UTF8
Bugs item #1103023, was opened at 2005-01-15 11:10
Message generated for change (Comment added) made by donut
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1103023&group_id=5470
Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Casey Crabb (airog)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input problem with readline and UTF8
Initial Comment:
Backspace doesn't remove all bytes of a multi-byte
UTF-8 character.
To reproduce the problem:
$ export LANG=en_US.UTF-8
$ python
Python 2.3.4 (#1, Jun 11 2004, 16:35:29)
[GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r3, ssp-3.3-7,
pie-8.5.3)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import readline
>>> raw_input() # ä, return
ä
'\xc3\xa4'
>>> raw_input() # ä, backspace, return
'\xc3'
>>>
A small C program does not have the same problem:
#include
#include
#include
#include
void pprint(const char *s);
int main(void) {
char *line;
for (;;) {
line = readline("> ");
if (!line)
break;
pprint(line);
free(line);
}
return 0;
}
void pprint(const char *s) {
while (*s) {
if (isprint(*s))
putchar(*s);
else
printf("\x%x", *s & 0xff);
s++;
}
putchar('\n');
}
--
Comment By: Matthew Mueller (donut)
Date: 2005-02-07 00:36
Message:
Logged In: YES
user_id=65253
Hi, it looks like this might be the same problem already
fixed in "[ 914291 ] Fix readline for utf-8 locales", but
your python version is from before the fix went in. Can you
try again with python 2.4, or 2.3.5 (when it is released, or
the release23-maint branch of cvs)?
Also, you could check if python's readline.so is linked to
an older version of libreadline.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1103023&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
