[issue12567] curses implementation of Unicode is wrong in Python 3

Ned Deily added the comment: See also Issue15037 which documents a broken curses.unget_wch and, hence, test_curses when Python is built with ncurses 5.7 or earlier. -- ___ Python tracker _

[issue12567] curses implementation of Unicode is wrong in Python 3

Ned Deily added the comment: It turns out that the Unicode support introduced by this issue didn't build correctly on OS X, either silently failing to build (explaining the problem seen by Nicholas) or causing a compile error (as seen in Issue14225). This should be working OK (as of 3.3.0b1)

[issue12567] curses implementation of Unicode is wrong in Python 3

Roundup Robot added the comment: New changeset 2035c5ad4239 by Ned Deily in branch 'default': Issue #14225: Fix Unicode support for curses (#12567) on OS X: http://hg.python.org/cpython/rev/2035c5ad4239 -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

Ned Deily added the comment: Nicholas, please open a new issue documenting which Python 3.3 you are using, from which python.org installer or the ./configure parameters if you built it yourself (and whether you supplied a version of GNU readline or used the Apple default of BSD libedit) and a

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > AttributeError: '_curses.curses window' object has no attribute 'get_wch' > is still being raised. "still"? Did it work before my last changes? Unicode functions of the (n)curses library are only available if the Python curses module is linked to libncurse

[issue12567] curses implementation of Unicode is wrong in Python 3

Nicholas Cole added the comment: Testing the Python3.3a2 build on OS X - the exception AttributeError: '_curses.curses window' object has no attribute 'get_wch' is still being raised. I don't have a Linux build I can easily test with. Is this a particular problem with the OS X build? -

[issue12567] curses implementation of Unicode is wrong in Python 3

Éric Araujo added the comment: Yes, it was only fixed for 3.3. -- nosy: +eric.araujo stage: -> committed/rejected ___ Python tracker ___ ___

[issue12567] curses implementation of Unicode is wrong in Python 3

Nicholas Cole added the comment: It looks to me as if the documentation in the release candidates for 2.7.3 and 3.2.3 haven't been updated to include the new curses fixes. Is that correct? -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > I wrote down when I set up the OpenIndiana buildbots Hum, please use the issue #13552 for curses issues on OpenIndiana/Solaris. > ... de funciones: "mvwchgat" y "wchgat" See issues #3786 and #13552 for this problem. > I installed ncurses ... I add export

[issue12567] curses implementation of Unicode is wrong in Python 3

Jesús Cea Avión added the comment: Victor, I have these notes I wrote down when I set up the OpenIndiana buildbots. Maybe can be useful to you: (compiling from source) """ * ncurses 5.7: Instalación estándar "./configure --with-shared --without-normal --enable-widec --without-cxx-binding".

[issue12567] curses implementation of Unicode is wrong in Python 3

Changes by STINNER Victor : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: The code has been commited. The remaining task is to fix OpenIndiana issues: see #13552. -- ___ Python tracker ___ ___

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > I am still concerned about the compilation warning in OpenIndiana buildbots > :-( I'm unable to reproduce the issue in my OpenIndiana VM: the compilaton of the _curses module fail, not because of Unicode, but because mvwchgat() function is missing => see t

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: New changeset bf51e32b2a81 by Victor Stinner in branch 'default': Issue #13415: test_curses skips unencodable characters http://hg.python.org/cpython/rev/bf51e32b2a81 (Oops, I copy-pasted the issue number from my previous commit, and the issue number was wrong

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: Compile output on OpenSolaris: Consider setting $PYTHONHOME to [:] ld: fatal: file /usr/local/lib/libncursesw.so: wrong ELF class: ELFCLASS32 ld: fatal: file processing errors. No output written to build/lib.solaris-2.11-i86pc-3.3-pydebug/readline.so collect2:

[issue12567] curses implementation of Unicode is wrong in Python 3

Jesús Cea Avión added the comment: I am still concerned about the compilation warning in OpenIndiana buildbots :-( -- ___ Python tracker ___

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > This broke several Gentoo buildbots. setup.py is unable to locate correctly curses.h. I added a hack to always search in /usr/include/ncursesw/. The hack is needed on Ubuntu 11.10 if you only have libncursesw5-dev but not libncursesw-dev for example. -

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: New changeset 919259054621 by Victor Stinner in branch 'default': Issue #13415: Help to locate curses.h when _curses module is linked to ncursesw http://hg.python.org/cpython/rev/919259054621 (Oops, wrong issue number, again) -- __

[issue12567] curses implementation of Unicode is wrong in Python 3

Antoine Pitrou added the comment: This broke several Gentoo buildbots. -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list m

[issue12567] curses implementation of Unicode is wrong in Python 3

Roundup Robot added the comment: New changeset c3581ca21a57 by Victor Stinner in branch 'default': Issue #12567: The curses module uses Unicode functions for Unicode arguments http://hg.python.org/cpython/rev/c3581ca21a57 -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

Changes by John Feuerstein : -- nosy: +john.feuerstein ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue12567] curses implementation of Unicode is wrong in Python 3

Changes by Petri Lehtinen : -- nosy: +petri.lehtinen ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue12567] curses implementation of Unicode is wrong in Python 3

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue12567] curses implementation of Unicode is wrong in Python 3

Roundup Robot added the comment: New changeset 786668a4fb6b by Victor Stinner in branch 'default': Issue #12567: Fix curses.unget_wch() tests http://hg.python.org/cpython/rev/786668a4fb6b -- ___ Python tracker ___

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > The LC_CTYPE locale should maybe be set temporary to > the current locale (""), as does locale.getpreferredencoding(). See also issue #6203. -- ___ Python tracker _

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: I'm not sure that it is correct to call nl_langinfo(CODESET) to get the locale encoding. The LC_CTYPE locale should maybe be set temporary to the current locale (""), as does locale.getpreferredencoding(). Or maybe better, locale.getpreferredencoding() should

[issue12567] curses implementation of Unicode is wrong in Python 3

Roundup Robot added the comment: New changeset b1e03d10391e by Victor Stinner in branch 'default': Issue #12567: Add curses.unget_wch() function http://hg.python.org/cpython/rev/b1e03d10391e -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > what needs to happen for them to move forward? I would like a review of curses_unicode.patch. -- ___ Python tracker ___

[issue12567] curses implementation of Unicode is wrong in Python 3

Nicholas Cole added the comment: There are now several bugs dealing with related issues here. Are we any closer to a solution to any of them? The suggested patches look like a good idea - what needs to happen for them to move forward? -- ___ Pyth

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: See also #10570. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue12567] curses implementation of Unicode is wrong in Python 3

Nadeem Vawda added the comment: Ack sorry, forgot to give context - my machine doesn't have libncursesw, so the curses module failed to build before that commit (with or without the patch applied). -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

Nadeem Vawda added the comment: Following d98b5e0f0862, I have been able to successfully build the curses module with curses_unicode.patch applied. -- nosy: +nadeem.vawda ___ Python tracker ___

[issue12567] curses implementation of Unicode is wrong in Python 3

Roundup Robot added the comment: New changeset d98b5e0f0862 by Nadeem Vawda in branch 'default': Fix build error in _curses module when not using libncursesw. http://hg.python.org/cpython/rev/d98b5e0f0862 -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: See also #6755 (curses.get_wch). -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubs

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: Using curses_unicode.patch: - without ncursesw: addch('é') raises an OverflowError because 'é'.encode('UTF-8') is 2 bytes and not 1 byte - with ncursesw: the charset is displayable character depends on the locale encoding (e.g. € cannot be printed with ISO-

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: Patch the _curses module to improve Unicode support: - add an encoding attribute to a window (only visible in C): read the locale encoding - encode a character and a character string to the window encoding if the ncursesw library is NOT used - addch(), add

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: > by the way: do all platforms have wide character functions? See msg140408 and msg140409: Antoine Pitrou (OS=Mageia 1) and some buildbots don't have get_wch(). -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

Nicholas Cole added the comment: I think that some platforms do not have wide character support, though I could be wrong. The FAQ here: http://invisible-island.net/ncurses/ncurses.faq.html has a list of those that do and those that don't, but I don't know how up to date it is. --

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: Oh, by the way: do all platforms have wide character functions? I don't see any failure on our Python 3.x buildbots, but test_curses is skipped on many buildbots. -- ___ Python tracker

[issue12567] curses implementation of Unicode is wrong in Python 3

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue12567] curses implementation of Unicode is wrong in Python 3

STINNER Victor added the comment: getkey.patch fixes window.getkey(): use get_wch() instead of getch() to handle correctly non-ASCII characters. I tested with the key é (U+00E9) with ISO-8859-1 and UTF-8 locale encoding: getkey() gives the expected result (but addstr is unable to display it,

[issue12567] curses implementation of Unicode is wrong in Python 3

New submission from STINNER Victor : curses functions accepting strings encode implicitly character strings to UTF-8. This is wrong. We should add a function to set the encoding (see issue #6745) or use the wide character C functions. I don't think that UTF-8 is the right default encoding, I s