[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Terry J. Reedy added the comment: IDLE avoids the problem of calculating a location for a '^' below the bad line by instead asking tk to give the marked character (and maybe more) a 'ERROR' tag, which shows as a red background. So it marks the '$' of 'A_I_U_E_O$' and the 'alid' slice of 'in

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Alexander Belopolsky added the comment: The original problem is still present Python 3.5.0a0 (default:5313b4c0bb6c, Sep 30 2014, 18:55:45) >>> A_I_U_E_O$ = None File "", line 1 A_I_U_E_O$ = None ^ SyntaxError: invalid syntax Replace A_I_U_E_O above with the Japanese script. I g

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: The issue #10384 has been marked as a duplicate of this issue: it's a similar issue, identifier which contains invisible character. -- ___ Python tracker __

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Serhiy Storchaka : -- assignee: serhiy.storchaka -> stage: patch review -> needs patch ___ Python tracker ___ ___ Python-bu

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Roundup Robot added the comment: New changeset eb7565c212f1 by Serhiy Storchaka in branch '3.3': Issue #2382: SyntaxError cursor "^" now is written at correct position in most http://hg.python.org/cpython/rev/eb7565c212f1 New changeset ea34b2b0b8ae by Serhiy Storchaka in branch 'default': Issue

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Serhiy Storchaka added the comment: If no one complain I'll commit last patch tomorrow. -- assignee: -> serhiy.storchaka stage: -> patch review type: -> behavior versions: +Python 3.4 -Python 3.2 ___ Python tracker

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Serhiy Storchaka : Removed file: http://bugs.python.org/file27506/adjust_offset-3.3.patch ___ Python tracker ___ ___ Python-bugs-lis

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Serhiy Storchaka added the comment: Added tests. I think it will be worth apply this patch which fixes the issue for most Europeans and than continue working on the issue of wide characters. -- Added file: http://bugs.python.org/file31874/adjust_offset_2.patch _

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Alexander Belopolsky added the comment: Serhiy's patch is lacking tests, but it passes the test I proposed at #10382 at attaching here. -- Added file: http://bugs.python.org/file30534/test.py ___ Python tracker ___

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Alexander Belopolsky added the comment: haypo> The purpose of this issue is to handle CJK characters taking 2 haypo> columns instead of 1 in a terminal, or did I misunderstand it? That's the other half of the problem, but the more common issue is misplaced caret when non-ascii characters are p

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: > This patch fixes a half of the issue - working with non-ascii > non-wide characters. The purpose of this issue is to handle CJK characters taking 2 columns instead of 1 in a terminal, or did I misunderstand it? -- ___

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Serhiy Storchaka added the comment: Here is a patch upgraded to Python 3.3. It uses a little different approach and works with invalid encoded data. unicode_utf8size.patch is not needed. This patch fixes a half of the issue - working with non-ascii non-wide characters. It's enough for many peo

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Serhiy Storchaka : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Petri Lehtinen added the comment: What's the status of this issue? FWIW, this is not only a problem with east asian characters: >>> ä äää File "", line 1 ä äää ^ SyntaxError: invalid syntax -- nosy: +petri.lehtinen versions: +Python 3.2, Python 3.3 -Python 3.0 _

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: I just created the issue #12568 for unicode_width.patch. -- ___ Python tracker ___ ___ Python-bugs-l

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Ezio Melotti : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: Comments about my own patches. unicode_width.patch: * error messages should be improved: ValueError("Unable to compute string width") for Windows IOError(strerror(errno)) otherwise adjust_offset.patch: * format_exception_only() from Lib/traceback.py ma

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by STINNER Victor : Added file: http://bugs.python.org/file13359/print_exception.patch ___ Python tracker ___ ___ Python-bugs-list mail

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by STINNER Victor : Added file: http://bugs.python.org/file13358/adjust_offset.patch ___ Python tracker ___ ___ Python-bugs-list mailin

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by STINNER Victor : Added file: http://bugs.python.org/file13357/unicode_width.patch ___ Python tracker ___ ___ Python-bugs-list mailin

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: For an easier review, I splitted my patch in multiple small patches: - unicode_utf8size.patch: create _PyUnicode_UTF8Size() function: Number of bytes needed to encode the unicode character as UTF-8 - unicode_width.patch: create PyUnicode_Width(): Number of co

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: Proof of concept of patch fixing this issue: - parse_syntax_error() reads the text line into a PyUnicodeObject* instead of a "const char**" - create utf8_to_unicode_offset(): convert byte offset to a number of characters. The Python version should be somethi

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

David W. Lambert added the comment: Resolution of this may be applicable to Issue3446 as well. "center, ljust and rjust are inconsistent with unicode parameters" -- nosy: +LambertDW ___ Python tracker _

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor added the comment: This issue is a problem of units. The error text is an utf8 *byte* string and offset is a number of *bytes*. The goal is to get the text *width* of a *character* string. We have to: 1- convert offset from bytes number to character number 2- get the error mes

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: Experimental patch was experimental, wcswidth(3) returns 1 for East Asian Ambiguous character. debian:~/python-dev/py3k# ./python /mnt/windows/a.py File "/mnt/windows/a.py", line 3 "♪xÅx" abc ^ should point 'c'. And anot

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: >At least my "one unicode char is one space" suggestion corrects the case >of Western languages, and all messages with single-width characters. I'm not happy with this solution. ;-( >Doesn't the exact width depend on >the terminal capab

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

STINNER Victor <[EMAIL PROTECTED]> added the comment: See also a related issue: issue3975. -- nosy: +haypo ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: This seems to be a difficult problem. Doesn't the exact width depend on the terminal capabilities? and fonts, and combining diacritics... An easy way to put the caret at the same exact position is to repeat the beginning of the line u

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: Amaury, if doing so, the cursor will shift left by 5 columns on my environment like this, no? ("あ" requires 2 columns for example) print "あいうえお" ^ ___ Python tracker <[EMAIL PROTECTE

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: For the moment, I'd suggest that one unicode character has a the same with as the space character, assuming that stdout.encoding correctly matches the terminal. Then the C implementation could do something similar to the statements I ad

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: You are right, this issue is more difficult than I thought... I found wcswidth(3), if this function is available we can use this function, but unfortunately there is no such function in VC6 and this function is meaningless on cygwn, so I ca

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: I think that your patch works only for terminals where one byte of the encoded text is displayed as one character on the terminal. This is not true for utf-8 terminals, for example. In the attached patch, I tried to write some unit te

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: Patch revised. -- components: +Interpreter Core -None Added file: http://bugs.python.org/file11548/py3k_adjust_cursor_at_syntax_error.patch ___ Python tracker <[EMAIL PROTECTED]>

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Hirokazu Yamamoto <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file9786/fix.patch ___ Python tracker <[EMAIL PROTECTED]> ___ ___

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Changes by Hirokazu Yamamoto <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file9723/experimental.patch __ Tracker <[EMAIL PROTECTED]> __ ___

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: > (I assumed get_length_in_bytes(f, " ", 1) == 1 but I'm not sure > this is always true in other platforms. Probably nicer and more > general solution may exist) This assumption still lives, but I cannot find better solution. I'm thinkin

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

Hirokazu Yamamoto <[EMAIL PROTECTED]> added the comment: > I tried to fix this problem, but I'm not sure how to fix this. Quick observation... /// // Possible Solution 1. Convert err->text to console compatible encoding (not to source encoding like in python2.x)

[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

New submission from Hirokazu Yamamoto <[EMAIL PROTECTED]>: Hello. I found another problem related to issue2301. SyntaxError cursor "^" is shifted when multibyte characters are in line (before "^"). I think this is because err->text is stored as UTF-8 which requires 3 bytes for multibyte characte