[issue1813] Codec lookup failing under turkish locale

Changes by Arfrever Frehtes Taifersar Arahesis : -- status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: I've upgraded the Fedora buildbot to Fedora-16. The specific glibc workaround should not be necessary any more. So the test will now fail again on all systems that a) have the bug and b) the tr_Tr locale. -- ___ Pytho

[issue1813] Codec lookup failing under turkish locale

Roundup Robot added the comment: New changeset a55ffb6c1993 by Stefan Krah in branch '3.2': Issue #1813: Revert workaround for a glibc bug on the Fedora buildbot. http://hg.python.org/cpython/rev/a55ffb6c1993 New changeset 4244e4348362 by Stefan Krah in branch 'default': Issue #1813: merge chan

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: https://bugzilla.redhat.com/show_bug.cgi?id=726536 claims that the glibc issue (which is relevant for skipping the test case) is fixed in glibc-2.14.90-8. I suspect the only way of running the test case reliably is whitelisting a couple of known good glibc version

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: > As someone pointed out on python-dev, if this isn't fixable then it > should be an expected failure, not a skip. The Python bug is fixed, the problem is apparently some libcs have the same bug as we did... > One question is, is there any platform on which th

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: [Re-opening to fix the skips] Yes, the test works on: Ubuntu Lucid (libc-2.11.1), OpenSUSE (libc-2.11.1), FreeBSD-8.2 Failure: Fedora 14 (libc-2.13), Debian lenny (libc-2.7), Gentoo (libc-2.13-r2) So perhaps this test should be marked as expected failure

[issue1813] Codec lookup failing under turkish locale

R. David Murray added the comment: On Tue, 02 Aug 2011 12:12:37 +0200, Stefan Krah wrote: > I suspect many buildbots are green because they don't have tr_TR and > tr_TR.iso8859-9 installed. This is true for my Gentoo buildbots. Once we've figured out the best way to handle this, I'll fix that

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: As I wrote on python-dev, this test also fails on Debian lenny, which has the same setlocale() bug as Fedora. So, indeed the test should be skipped on a multitude of platforms. -- ___ Python tracker

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: Unrelated to the Fedora issue: The test is currently skipped on the FreeBSD bot, but completes successfully with: diff -r 0b52b6f1bfab Lib/test/test_locale.py --- a/Lib/test/test_locale.py Tue Aug 02 10:16:45 2011 +0200 +++ b/Lib/test/test_locale.py Tue Aug 02

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: Fedora bug report: https://bugzilla.redhat.com/show_bug.cgi?id=726536 -- ___ Python tracker ___ ___ Py

[issue1813] Codec lookup failing under turkish locale

R. David Murray added the comment: I'm seeing this test failure in Gentoo, as well. -- nosy: +r.david.murray ___ Python tracker ___ __

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: Yes, it's a bug. This works: #include #include int main(void) { char *s; printf("%s\n", setlocale(LC_CTYPE, "tr_TR.ISO8859-9")); printf("%s\n", setlocale(LC_CTYPE, NULL)); s = setlocale(LC_CTYPE, "tr_TR.ISO8859-9"); printf("%s\n", s ? s : "nu

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: > Stefan Krah wrote: > > (gdb) p result = setlocale(category, "tr_TR.ISO8859-9") > > $8 = 0x0 > > (gdb) p result = setlocale(category, "tr_TR") > > $9 = 0x96d770 "tr_TR" > > (gdb) p locale > > $10 = 0x70f6a5b0 "tr_TR.ISO8859-9" > > (gdb) > > Perhaps this i

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: Stefan Krah wrote: > (gdb) p result = setlocale(category, "tr_TR.ISO8859-9") > $8 = 0x0 > (gdb) p result = setlocale(category, "tr_TR") > $9 = 0x96d770 "tr_TR" > (gdb) p locale > $10 = 0x70f6a5b0 "tr_TR.ISO8859-9" > (gdb) Perhaps this is a bug in Fedora's set

[issue1813] Codec lookup failing under turkish locale

Stefan Krah added the comment: The Fedora bot fails because here ... locale.setlocale(locale.LC_CTYPE, loc) loc = ('tr_TR', 'ISO8859-9'), and apparently setlocale can only handle "tr_TR", but not "tr_TR.ISO8859-9": 144 if (locale) { 145 /* set locale */ 146

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: Finally fixed in 2.7, 3.2, 3.3! -- resolution: -> fixed stage: -> committed/rejected status: open -> closed versions: +Python 3.3 -Python 2.6, Python 3.1 ___ Python tracker _

[issue1813] Codec lookup failing under turkish locale

Roundup Robot added the comment: New changeset 739958134fe5 by Antoine Pitrou in branch '2.7': Issue #1813: Fix codec lookup and setting/getting locales under Turkish locales. http://hg.python.org/cpython/rev/739958134fe5 -- ___ Python tracker

[issue1813] Codec lookup failing under turkish locale

Roundup Robot added the comment: New changeset 92d02de91cc9 by Antoine Pitrou in branch '3.2': Issue #1813: Fix codec lookup under Turkish locales. http://hg.python.org/cpython/rev/92d02de91cc9 New changeset a77a4df54b95 by Antoine Pitrou in branch '3.2': Add a test for issue #1813: getlocale()

[issue1813] Codec lookup failing under turkish locale

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue1813] Codec lookup failing under turkish locale

STINNER Victor added the comment: The decimal module has been fixed in Python 2.7, 3.2 and 3.3 for Turkish local: issue #11830. -- ___ Python tracker ___ ___

[issue1813] Codec lookup failing under turkish locale

Changes by Gökçen Eraslan : -- nosy: +Gökçen.Eraslan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue1813] Codec lookup failing under turkish locale

Marc-Andre Lemburg added the comment: Looking at this again, I think we should change the codec registry C code to use Py_TOLOWER() and the encoding search function code to use the .translate() approach that Antoine suggested. -- ___ Python tracker

[issue1813] Codec lookup failing under turkish locale

Dirkjan Ochtman added the comment: We've included this patch in Gentoo for about two years now. Can we get some discussion going on doing something like this? -- nosy: +djc ___ Python tracker _

[issue1813] Codec lookup failing under turkish locale

STINNER Victor added the comment: There is also a locale normalization function in unicodeobject.c: normalize_encoding(). This function uses "if (ISUPPER(*e)) *l++ = TOLOWER(*e++);" which uses the Python, *locale-independent*, implementation of ctype. We should maybe use the ISUPPER / TOLOWE

[issue1813] Codec lookup failing under turkish locale

Mark Lawrence added the comment: Does anyone know if this was discussed on python-dev? I've tried searching the archives and didn't find anything, but that's not to say it isn't there. -- nosy: +BreamoreBoy ___ Python tracker

[issue1813] Codec lookup failing under turkish locale

Changes by Antoine Pitrou : -- nosy: +haypo versions: +Python 2.7, Python 3.1, Python 3.2 ___ Python tracker ___ ___ Python-bugs-list m

[issue1813] Codec lookup failing under turkish locale

Changes by Jakub Wilk : -- nosy: +jwilk ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/m

[issue1813] Codec lookup failing under turkish locale

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: Sean: I'd suggest to discuss this on python-dev. Note that even if we do use Unicode for the cases in question, the Turkish locale will still pose a problem - see #1528802 for a discussion. __ Tracker <[EM

[issue1813] Codec lookup failing under turkish locale

Sean Reifschneider <[EMAIL PROTECTED]> added the comment: Marc-Andre: How should we proceed with this bug? Discuss on python-dev or c.l.python? -- assignee: -> lemburg keywords: +patch nosy: +jafo priority: -> normal __ Tracker <[EMAIL PROTECTED]>

[issue1813] Codec lookup failing under turkish locale

Marc-Andre Lemburg added the comment: I agree that it's a bit unfortunate that the 8-bit string APIs in Python use the locale aware C functions per default (this should really be reversed: there should be locale-aware .upper() and .lower() methods and the the standard ones should work just like t

[issue1813] Codec lookup failing under turkish locale

Changes by Antoine Pitrou: -- versions: +Python 2.6 -Python 2.5 __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-list mailing list Unsubscribe

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: Even if we don't fix all uses of (?to)(lower|upper) in the source tree, I think it's important that codec and locale lookup work properly when the current locale defines non-latin case folding for latin characters. Here is a patch. Perhaps also the str type shou

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: As for the .upper() and .lower() methods, they are used in quite a bunch of standard library modules :-/... Lib/base64.py Lib/BaseHTTPServer.py Lib/bsddb/test/test_compare.py Lib/bsddb/test/test_dbobj.py Lib/CGIHTTPServer.py Lib/cgi.py Lib/compiler/ast.py Lib/Co

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: The C library's tolower() and toupper() are used in a handful of source files. It might make sense to replace some of those calls with ascii-only versions of the corresponding functions. Modules/_sre.c:return ((ch) < 256 ? (unsigned int)tolower((ch)) : ch);

[issue1813] Codec lookup failing under turkish locale

Árni Már Jónsson added the comment: There is more to this bug than appears. I'm guessing that the name mangling code in locale (e.g. the normalizing code) is locale dependent. See this example: #!/usr/bin/python2.5 import locale print 'TR', locale.normalize('tr') print locale.setlocale(loca

[issue1813] Codec lookup failing under turkish locale

Antoine Pitrou added the comment: I can confirm this on SVN trunk on a Mandriva system. -- nosy: +pitrou __ Tracker <[EMAIL PROTECTED]> __ ___ P

[issue1813] Codec lookup failing under turkish locale

Changes by Árni Már Jónsson: -- components: +Library (Lib) -Interpreter Core __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-list mailing list

[issue1813] Codec lookup failing under turkish locale

New submission from Árni Már Jónsson: When switching to a turkish locale, the codecs registry fails on a codec lookup which worked before the locale change. This happens when the codec name contains an uppercase 'I'. What happens, is just before doing a cache lookup, the string is normalized, wh