[ python-Bugs-1080864 ] locale.py doesn't recognize valid locale setting
Bugs item #1080864, was opened at 2004-12-07 21:23
Message generated for change (Comment added) made by childsplay
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1080864&group_id=5470
Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: stas Z (childsplay)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.py doesn't recognize valid locale setting
Initial Comment:
[EMAIL PROTECTED]:~$ locale
LANG=nb_NO
[...]
[EMAIL PROTECTED]:~$ python
Python 2.3.4 (#2, Sep 24 2004, 08:39:09)
[GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.3/locale.py", line 346, in
getdefaultlocale
return _parse_localename(localename)
File "/usr/lib/python2.3/locale.py", line 280, in
_parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: nb_NO
>>>
--
>Comment By: stas Z (childsplay)
Date: 2004-12-09 10:27
Message:
Logged In: YES
user_id=638376
I agree that "default" would probably be called "preferred".
@loewis:
a)
I agree with your point of view but I as a developer I just
want to get the current locale in use and locale.py serves
that purpose in a platform independant way. The
"never-ending maintenance problem" is the result of the
locale horror we all have to live with until there's a final
solution/standard for it.
b)
Agree, I now understand your "getdefaultlocale ..
inherently broken" comment. But we still have a locale
module that supports some of the valid locales but not all,
which is (IMO) worse then having none at all.
BTW: getting the current locale to get a platform
independant language setting, should perhaps be part of
gettext.py instead of locale.py?
--
Comment By: Martin v. Löwis (loewis)
Date: 2004-12-08 22:28
Message:
Logged In: YES
user_id=21627
MAL: that doesn't answer my question, though: In what sense
does getdefaultlocale get a "default" locale? default as
opposed to what custom setting?
childsplay: the problem with adding additional aliases is
a) we can never hope to get a complete list of locales, so
this is a never-ending maintenance problem, and
b) the dictionary might be wrong on some systems. E.g.
sometimes, 'nb' might denote an ISO-8859-15 locale, or a
UTF-8 locale (e.g. when UTF-8 becomes the standard encoding
on Unix some day). If so, Python will silently compute an
incorrect default - in particular wrt. the encoding of the
"default" locale.
--
Comment By: stas Z (childsplay)
Date: 2004-12-08 16:09
Message:
Logged In: YES
user_id=638376
This is what I've put into /python2.3/locale.py:
locale_alias = {
...
'bokmål': 'nb_NO.ISO8859-1',
'nb': 'nb_NO.ISO8859-1',
'nb_no':'nb_NO.ISO8859-1',
'nynorsk': 'nn_NO.ISO8859-1',
'nn': 'nn_NO.ISO8859-1',
'nn_no':'nn_NO.ISO8859-1',
}
I have tested it on a number of apps and it fixes the problem.
--
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-12-08 14:06
Message:
Logged In: YES
user_id=38388
Please provide some authorative source which describes the
locale your are using (nb_NO) and the commonly used encoding
for that locale (see the existing dictionary in locale.py).
--
Comment By: stas Z (childsplay)
Date: 2004-12-08 13:29
Message:
Logged In: YES
user_id=638376
The reason I use getdefaultlocale(), is to get a platform
independant way of getting the systems locale setting.
The biggest advantage is that on Windows a "Linux like"
locale is returnt so that I can use the same language
support stuff on all
platforms. (Win,Linux,OSX).
Besides, what's the problem of adding the missing locale?
--
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-12-08 10:05
Message:
Logged In: YES
user_id=38388
The LANG (and other similar OS environment variables) define
what the user wishes to see being used as locale in the
applications that are started in that environment. See the
setlocale man page for details.
On some OSes such as Windows these settings are stored
differently, which is why the locale module has provisions
for finding these settings (thanks to Fredrik).
--
Comment By: Martin v. Löwis (loewis)
Date: 2004-12-08 08:04
Message:
Logged In: YES
user_id=21
[ python-Bugs-1080864 ] locale.py doesn't recognize valid locale setting
Bugs item #1080864, was opened at 2004-12-07 21:23
Message generated for change (Comment added) made by lemburg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1080864&group_id=5470
Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: stas Z (childsplay)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.py doesn't recognize valid locale setting
Initial Comment:
[EMAIL PROTECTED]:~$ locale
LANG=nb_NO
[...]
[EMAIL PROTECTED]:~$ python
Python 2.3.4 (#2, Sep 24 2004, 08:39:09)
[GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.3/locale.py", line 346, in
getdefaultlocale
return _parse_localename(localename)
File "/usr/lib/python2.3/locale.py", line 280, in
_parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: nb_NO
>>>
--
>Comment By: M.-A. Lemburg (lemburg)
Date: 2004-12-09 10:42
Message:
Logged In: YES
user_id=38388
Martin:
"default" as opposed to whatever locale setting is currently
active for the program, i.e. the locale setting the program
would see after a single call to setlocale(LC_ALL, "") right
after the start of the program.
getdefaultlocale() mimics the lookup mechanism of
setlocale(LC_ALL, "").
The fact that the alias table may sometimes not give the
correct encoding is not a fault of Python or the table - if
the user wants to see a different encoding used as default
encoding for the set locale, then the user should include
that information in the LANG (or other) OS environment
variable of the process running the Python program.
Note that this is different than the "preferred" encoding
which a user can set in a window manager (KDE or Gnome) or
browser. Those settings are restricted to certain
application spaces. getdefaultencoding() is targetted at the
OS level setting which may be different from e.g. a KDE
setting (think a program running in a shell vs. a KDE
application run by a user).
--
Comment By: stas Z (childsplay)
Date: 2004-12-09 10:27
Message:
Logged In: YES
user_id=638376
I agree that "default" would probably be called "preferred".
@loewis:
a)
I agree with your point of view but I as a developer I just
want to get the current locale in use and locale.py serves
that purpose in a platform independant way. The
"never-ending maintenance problem" is the result of the
locale horror we all have to live with until there's a final
solution/standard for it.
b)
Agree, I now understand your "getdefaultlocale ..
inherently broken" comment. But we still have a locale
module that supports some of the valid locales but not all,
which is (IMO) worse then having none at all.
BTW: getting the current locale to get a platform
independant language setting, should perhaps be part of
gettext.py instead of locale.py?
--
Comment By: Martin v. Löwis (loewis)
Date: 2004-12-08 22:28
Message:
Logged In: YES
user_id=21627
MAL: that doesn't answer my question, though: In what sense
does getdefaultlocale get a "default" locale? default as
opposed to what custom setting?
childsplay: the problem with adding additional aliases is
a) we can never hope to get a complete list of locales, so
this is a never-ending maintenance problem, and
b) the dictionary might be wrong on some systems. E.g.
sometimes, 'nb' might denote an ISO-8859-15 locale, or a
UTF-8 locale (e.g. when UTF-8 becomes the standard encoding
on Unix some day). If so, Python will silently compute an
incorrect default - in particular wrt. the encoding of the
"default" locale.
--
Comment By: stas Z (childsplay)
Date: 2004-12-08 16:09
Message:
Logged In: YES
user_id=638376
This is what I've put into /python2.3/locale.py:
locale_alias = {
...
'bokmål': 'nb_NO.ISO8859-1',
'nb': 'nb_NO.ISO8859-1',
'nb_no':'nb_NO.ISO8859-1',
'nynorsk': 'nn_NO.ISO8859-1',
'nn': 'nn_NO.ISO8859-1',
'nn_no':'nn_NO.ISO8859-1',
}
I have tested it on a number of apps and it fixes the problem.
--
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-12-08 14:06
Message:
Logged In: YES
user_id=38388
Please provide some authorative source which describes the
locale your are using (nb_NO) and the commonly used encoding
for that locale (see the existing dictionary in locale.py).
---
[ python-Bugs-1080864 ] locale.py doesn't recognize valid locale setting
Bugs item #1080864, was opened at 2004-12-07 21:23
Message generated for change (Comment added) made by lemburg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1080864&group_id=5470
Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: stas Z (childsplay)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.py doesn't recognize valid locale setting
Initial Comment:
[EMAIL PROTECTED]:~$ locale
LANG=nb_NO
[...]
[EMAIL PROTECTED]:~$ python
Python 2.3.4 (#2, Sep 24 2004, 08:39:09)
[GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.3/locale.py", line 346, in
getdefaultlocale
return _parse_localename(localename)
File "/usr/lib/python2.3/locale.py", line 280, in
_parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: nb_NO
>>>
--
>Comment By: M.-A. Lemburg (lemburg)
Date: 2004-12-09 10:56
Message:
Logged In: YES
user_id=38388
childsplay (I wish people would use real names on SF...):
We can add the aliases you gave below, but we need some URLs
to add as reference. Please provide this information, so
that we can document that the aliases are in common use and
why iso-8859-1 is their usually used encoding.
Thanks.
--
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-12-09 10:42
Message:
Logged In: YES
user_id=38388
Martin:
"default" as opposed to whatever locale setting is currently
active for the program, i.e. the locale setting the program
would see after a single call to setlocale(LC_ALL, "") right
after the start of the program.
getdefaultlocale() mimics the lookup mechanism of
setlocale(LC_ALL, "").
The fact that the alias table may sometimes not give the
correct encoding is not a fault of Python or the table - if
the user wants to see a different encoding used as default
encoding for the set locale, then the user should include
that information in the LANG (or other) OS environment
variable of the process running the Python program.
Note that this is different than the "preferred" encoding
which a user can set in a window manager (KDE or Gnome) or
browser. Those settings are restricted to certain
application spaces. getdefaultencoding() is targetted at the
OS level setting which may be different from e.g. a KDE
setting (think a program running in a shell vs. a KDE
application run by a user).
--
Comment By: stas Z (childsplay)
Date: 2004-12-09 10:27
Message:
Logged In: YES
user_id=638376
I agree that "default" would probably be called "preferred".
@loewis:
a)
I agree with your point of view but I as a developer I just
want to get the current locale in use and locale.py serves
that purpose in a platform independant way. The
"never-ending maintenance problem" is the result of the
locale horror we all have to live with until there's a final
solution/standard for it.
b)
Agree, I now understand your "getdefaultlocale ..
inherently broken" comment. But we still have a locale
module that supports some of the valid locales but not all,
which is (IMO) worse then having none at all.
BTW: getting the current locale to get a platform
independant language setting, should perhaps be part of
gettext.py instead of locale.py?
--
Comment By: Martin v. Löwis (loewis)
Date: 2004-12-08 22:28
Message:
Logged In: YES
user_id=21627
MAL: that doesn't answer my question, though: In what sense
does getdefaultlocale get a "default" locale? default as
opposed to what custom setting?
childsplay: the problem with adding additional aliases is
a) we can never hope to get a complete list of locales, so
this is a never-ending maintenance problem, and
b) the dictionary might be wrong on some systems. E.g.
sometimes, 'nb' might denote an ISO-8859-15 locale, or a
UTF-8 locale (e.g. when UTF-8 becomes the standard encoding
on Unix some day). If so, Python will silently compute an
incorrect default - in particular wrt. the encoding of the
"default" locale.
--
Comment By: stas Z (childsplay)
Date: 2004-12-08 16:09
Message:
Logged In: YES
user_id=638376
This is what I've put into /python2.3/locale.py:
locale_alias = {
...
'bokmål': 'nb_NO.ISO8859-1',
'nb': 'nb_NO.ISO8859-1',
'nb_no':'nb_NO.ISO8859-1',
'nynorsk': 'nn_NO.ISO8859-1',
'nn': 'nn_NO.ISO8859-1',
'nn_no':'nn_NO.ISO8859-1',
...
[ python-Bugs-1075427 ] urllib2.HTTPBasicAuthHandler problem with [HOST]:[PORT]
Bugs item #1075427, was opened at 2004-11-29 19:06 Message generated for change (Comment added) made by mmzeeman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1075427&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: O-Zone (o-zone2004) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2.HTTPBasicAuthHandler problem with [HOST]:[PORT] Initial Comment: I've encountered problems authenticating to an host with this piece of code: passman = urllib2.HTTPPasswordMgrWithDefaultRealm() passman.add_password(None, theurl, username, password) authhandler = urllib2.HTTPBasicAuthHandler(passman) opener = urllib2.build_opener(authhandler) urllib2.install_opener(opener) req = urllib2.Request(theurl) try: handle = urllib2.urlopen(req) if i specify an URL like: http://[MyHostIP]:[Port] auth helper never authorize me on the host. It work perfectly if i don't specify ANY port in the URL. Regards, Michele "O-Zone" Pinassi -- Comment By: MM Zeeman (mmzeeman) Date: 2004-12-09 11:33 Message: Logged In: YES user_id=350634 This is actually the same problem as reported in bug 974757, which has a patch attached to it. The problem is that with basic authentication only the hostname + portnumber is passed to find_user_password. This method works if there is no port number when a netloc is used. It looks like the answer from reduce_uri is wrong when port numbers are used. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1075427&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-09 08:08 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-09 08:08 Message generated for change (Settings changed) made by ajung You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None >Priority: 7 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-09 08:08 Message generated for change (Comment added) made by ajung You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 7 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- >Comment By: Andreas Jung (ajung) Date: 2004-12-09 08:12 Message: Logged In: YES user_id=11084 This error not only happens with u'\xc4', it happens with *any* string containing a character >0x7f. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-10 00:08 Message generated for change (Comment added) made by anthonybaxter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 7 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- >Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-10 01:04 Message: Logged In: YES user_id=29957 Please attach a test case that shows the failure. I can't reproduce it here with the information you've given. -- Comment By: Andreas Jung (ajung) Date: 2004-12-10 00:12 Message: Logged In: YES user_id=11084 This error not only happens with u'\xc4', it happens with *any* string containing a character >0x7f. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-09 13:08 Message generated for change (Comment added) made by dcjim You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 7 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- >Comment By: Jim Fulton (dcjim) Date: 2004-12-09 14:14 Message: Logged In: YES user_id=73023 I can't reproduce this: >>> ick = u'\xc4' >>> import struct >>> struct.pack(ick) Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\xc4' in position 0: ordinal not in range(128) >>> struct.pack(u'i', 1) '\x01\x00\x00\x00' The code that provked this in Zope is buggy., -- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-09 14:04 Message: Logged In: YES user_id=29957 Please attach a test case that shows the failure. I can't reproduce it here with the information you've given. -- Comment By: Andreas Jung (ajung) Date: 2004-12-09 13:12 Message: Logged In: YES user_id=11084 This error not only happens with u'\xc4', it happens with *any* string containing a character >0x7f. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-09 13:08 Message generated for change (Settings changed) made by dcjim You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 >Status: Closed >Resolution: Rejected Priority: 7 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- Comment By: Jim Fulton (dcjim) Date: 2004-12-09 14:14 Message: Logged In: YES user_id=73023 I can't reproduce this: >>> ick = u'\xc4' >>> import struct >>> struct.pack(ick) Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\xc4' in position 0: ordinal not in range(128) >>> struct.pack(u'i', 1) '\x01\x00\x00\x00' The code that provked this in Zope is buggy., -- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-09 14:04 Message: Logged In: YES user_id=29957 Please attach a test case that shows the failure. I can't reproduce it here with the information you've given. -- Comment By: Andreas Jung (ajung) Date: 2004-12-09 13:12 Message: Logged In: YES user_id=11084 This error not only happens with u'\xc4', it happens with *any* string containing a character >0x7f. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082085 ] PyString_AsString() segfaults when passed a unicode string
Bugs item #1082085, was opened at 2004-12-09 08:08 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 Category: Python Interpreter Core >Group: Not a Bug Status: Closed >Resolution: Invalid Priority: 7 Submitted By: Andreas Jung (ajung) Assigned to: Nobody/Anonymous (nobody) Summary: PyString_AsString() segfaults when passed a unicode string Initial Comment: If you pass a PyObject representation the unicode string u'\xc4' to PyString_AsString() then Python (2.3.4, 2.4.0) segfault. Famous last words of Python 2.4: Exception exceptions.UnicodeEncodeError: in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Famous last words of Python 2.4 (debug build): XXX undetected error Traceback (most recent call last): File "test.py", line 4, in ? print S.split(u'\xc4') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 0: ordinal not in range(128) [6545 refs] This bug has been reported first on the zope-dev list. I confirmed that this error does not only occur in Zope but also in a Python-only environment. -- >Comment By: Tim Peters (tim_one) Date: 2004-12-09 10:05 Message: Logged In: YES user_id=31435 For the record, this appeared to be due to an extension module doing PyString_AsString(name)[0] when name was a Unicode string containing a "high-bit" character. PyString_AsString(name) legitimately returned NULL, and bad stuff followed. -- Comment By: Jim Fulton (dcjim) Date: 2004-12-09 09:14 Message: Logged In: YES user_id=73023 I can't reproduce this: >>> ick = u'\xc4' >>> import struct >>> struct.pack(ick) Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\xc4' in position 0: ordinal not in range(128) >>> struct.pack(u'i', 1) '\x01\x00\x00\x00' The code that provked this in Zope is buggy., -- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-09 09:04 Message: Logged In: YES user_id=29957 Please attach a test case that shows the failure. I can't reproduce it here with the information you've given. -- Comment By: Andreas Jung (ajung) Date: 2004-12-09 08:12 Message: Logged In: YES user_id=11084 This error not only happens with u'\xc4', it happens with *any* string containing a character >0x7f. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082085&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1076233 ] distutils.core.setup() with unicode arguments broken
Bugs item #1076233, was opened at 2004-11-30 14:53
Message generated for change (Comment added) made by akuchling
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1076233&group_id=5470
Category: Distutils
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: A.M. Kuchling (akuchling)
Summary: distutils.core.setup() with unicode arguments broken
Initial Comment:
When using unicode arguments for distutils.core.setup()
running setup.py breaks with the following exception:
Traceback (most recent call last):
...
\Lib\distutils\command\sdist.py", line 430, in
make_release_tree
self.distribution.metadata.write_pkg_info(base_dir)
File "C:\Programme\Python24\Lib\distutils\dist.py", line
1047, in write_pkg_info
pkg_info.write('Author: %s\n' % self.get_contact() )
UnicodeEncodeError: 'ascii' codec can't encode
character u'\xf6' in position 16: ordinal not in range(128)
Changing the system default encoding to iso-8859-1
works around this problem.
--
>Comment By: A.M. Kuchling (akuchling)
Date: 2004-12-09 13:59
Message:
Logged In: YES
user_id=11375
Encoding was never considered. PEP 314 is an updated
version of PEP 241; it should specify an encoding (probably
UTF-8 is the most reasonable choice).
--
Comment By: Thomas Heller (theller)
Date: 2004-11-30 15:13
Message:
Logged In: YES
user_id=11105
I'm unusure about the allowed encoding in PKG-INFO, and I
cannot find anything in PEP 241. Andrew, since you are
listed as the author, can you comment?
bdist_wininst does accept unicode strings, and that may be
the reason that walter expects this to work for sdist also...
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1076233&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1080811 ] full test with all unicode text files
Bugs item #1080811, was opened at 2004-12-07 19:53 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1080811&group_id=5470 Category: None Group: Not a Bug Status: Open Resolution: None Priority: 5 Submitted By: Grzegorz Makarewicz (makaron) Assigned to: Nobody/Anonymous (nobody) Summary: full test with all unicode text files Initial Comment: samall bug? while performing full test on pythonthon core with all required files (unicode). This can be found when "python -u regrtest.py -uall" is executed - perhaps some encodings are not preserved - test_codecmaps_kr fails with message: "JOHAB.TXT not found, download from http://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/KSC/JOHAB.TXT " this file exists on my system and is placed in lib/test, as required. when its running as standalone test everything is OK. -- >Comment By: Walter Dörwald (doerwalter) Date: 2004-12-09 20:06 Message: Logged In: YES user_id=89016 I can't reproduce this on Linux. Putting JOHAB.TXT (and the other files) into the current directory (dist from a CVS checkout) and running "python Lib/test/regrtest.py" works. Putting all the files into Lib/test and running regrtest.py from there works too. Which version of Python are you using? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1080811&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1082487 ] font lock keyword regular expressions
Bugs item #1082487, was opened at 2004-12-09 16:43 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082487&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Robert Brown (robert-brown) Assigned to: Nobody/Anonymous (nobody) Summary: font lock keyword regular expressions Initial Comment: I've noticed that when I use Python mode (alpha 1) with font lock mode, "self" is highlighted in the following line: his_self() The problem appears to be a combination of using "\b" in the Python font lock regular expressions and my .emacs file, which does: (modify-syntax-entry ?\_ "_" py-mode-syntax-table) I do not experience similar problems with highlighting in C mode, but there "\<" and "\>" are used in the font lock regular expressions. Using these word delimiters instead of "\b" appears to fix the problem for me in Python mode. Please wrap keywords with "\<" and "\>" instead of with "\b" in the font lock regular expression. Thanks very much. Bob -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1082487&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1075902 ] ignore element format character for PyArg_ParseTuple
Feature Requests item #1075902, was opened at 2004-11-30 11:28 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1075902&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sean Proctor (sproctor) Assigned to: Nobody/Anonymous (nobody) Summary: ignore element format character for PyArg_ParseTuple Initial Comment: Some kind of option to indicate there is an element, but we don't care about its value would be nice. It could be _. then an example would be: PyArg_ParseTuple (tuple, "_i", &my_int); This indicates that there is another value before the int, but we don't care about it one way or the other. -- >Comment By: Martin v. Löwis (loewis) Date: 2004-12-10 00:21 Message: Logged In: YES user_id=21627 So just call the variable _: PyObject *_; PyArg_ParseTuple(tuple, "Oi", &_, &my_int); Given that this is possible today already, I expect that this RFE will be considered so minor that it won't be implemented anytime soon. It increases the learning curve for a nearly-unnoticable convenience. -- Comment By: Sean Proctor (sproctor) Date: 2004-12-09 06:37 Message: Logged In: YES user_id=43755 that would turn my example into PyArg_ParseTuple (tuple, "Oi", _, &my_int); I don't particularly care. I thought it was a good suggestion. To me _ looks like a blank. I wouldn't really want to use it in a format string for something besides an any/none scenario. Your reasoning about preserving format characters seems really weak. You've never had a tuple where you didn't care about the value of each element? Somewhat separate topic. Could you say "PyArg_ParseTuple (tuple, "Oi", NULL, &my_int);" ? I guess that would be a less elegant way of doing what I intended. -- Comment By: Tim Peters (tim_one) Date: 2004-12-09 06:04 Message: Logged In: YES user_id=31435 If you want to make C code look like ML, declare PyObject** _; and use _ in the argument list. I'm -1 on this: the case has almost never arisen in my experience, and the limited pool of format characters should be preserved for high-value uses. -- Comment By: Sean Proctor (sproctor) Date: 2004-12-09 04:35 Message: Logged In: YES user_id=43755 If I use O, then I need to have a variable to store it in. I think it's less readable. Like with pattern matching in ML, you can use a variable instead of _, but it's not obvious that you aren't going to use that value. With some kind of place holder, this is made clear, and I don't need to create a variable to store a value that I don't care about. -- Comment By: Martin v. Löwis (loewis) Date: 2004-12-08 22:42 Message: Logged In: YES user_id=21627 What's wrong with using O as the format character? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1075902&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1080864 ] locale.py doesn't recognize valid locale setting
Bugs item #1080864, was opened at 2004-12-07 21:23 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1080864&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: stas Z (childsplay) Assigned to: Nobody/Anonymous (nobody) Summary: locale.py doesn't recognize valid locale setting Initial Comment: [EMAIL PROTECTED]:~$ locale LANG=nb_NO [...] [EMAIL PROTECTED]:~$ python Python 2.3.4 (#2, Sep 24 2004, 08:39:09) [GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.getdefaultlocale() Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.3/locale.py", line 346, in getdefaultlocale return _parse_localename(localename) File "/usr/lib/python2.3/locale.py", line 280, in _parse_localename raise ValueError, 'unknown locale: %s' % localename ValueError: unknown locale: nb_NO >>> -- >Comment By: Martin v. Löwis (loewis) Date: 2004-12-10 00:26 Message: Logged In: YES user_id=21627 getdefaultencoding might be "targeted" at the OS level - but the implementation certainly is not. On the OS level, the C library will often use a different encoding after locale.setlocale(locale.LC_ALL,"") is called, compared to what getdefaultencoding returns. The approach takien inside getdefaultencoding is inherently flawed, and cannot possibly work in all cases; getpreferredencoding fixes that flaw. -- Comment By: M.-A. Lemburg (lemburg) Date: 2004-12-09 10:56 Message: Logged In: YES user_id=38388 childsplay (I wish people would use real names on SF...): We can add the aliases you gave below, but we need some URLs to add as reference. Please provide this information, so that we can document that the aliases are in common use and why iso-8859-1 is their usually used encoding. Thanks. -- Comment By: M.-A. Lemburg (lemburg) Date: 2004-12-09 10:42 Message: Logged In: YES user_id=38388 Martin: "default" as opposed to whatever locale setting is currently active for the program, i.e. the locale setting the program would see after a single call to setlocale(LC_ALL, "") right after the start of the program. getdefaultlocale() mimics the lookup mechanism of setlocale(LC_ALL, ""). The fact that the alias table may sometimes not give the correct encoding is not a fault of Python or the table - if the user wants to see a different encoding used as default encoding for the set locale, then the user should include that information in the LANG (or other) OS environment variable of the process running the Python program. Note that this is different than the "preferred" encoding which a user can set in a window manager (KDE or Gnome) or browser. Those settings are restricted to certain application spaces. getdefaultencoding() is targetted at the OS level setting which may be different from e.g. a KDE setting (think a program running in a shell vs. a KDE application run by a user). -- Comment By: stas Z (childsplay) Date: 2004-12-09 10:27 Message: Logged In: YES user_id=638376 I agree that "default" would probably be called "preferred". @loewis: a) I agree with your point of view but I as a developer I just want to get the current locale in use and locale.py serves that purpose in a platform independant way. The "never-ending maintenance problem" is the result of the locale horror we all have to live with until there's a final solution/standard for it. b) Agree, I now understand your "getdefaultlocale .. inherently broken" comment. But we still have a locale module that supports some of the valid locales but not all, which is (IMO) worse then having none at all. BTW: getting the current locale to get a platform independant language setting, should perhaps be part of gettext.py instead of locale.py? -- Comment By: Martin v. Löwis (loewis) Date: 2004-12-08 22:28 Message: Logged In: YES user_id=21627 MAL: that doesn't answer my question, though: In what sense does getdefaultlocale get a "default" locale? default as opposed to what custom setting? childsplay: the problem with adding additional aliases is a) we can never hope to get a complete list of locales, so this is a never-ending maintenance problem, and b) the dictionary might be wrong on some systems. E.g. sometimes, 'nb' might denote an ISO-8859-15 locale, or a UTF-8 locale (e.g. when UTF-8 becomes the standard encoding on Unix some day). If so, Python will silently compute an incorrect default - in particular wrt. the encoding of the "default
