[issue10329] trace.py and unicode in Python 3

Changes by Alexander Belopolsky : -- stage: test needed -> committed/rejected status: open -> closed ___ Python tracker ___ ___ Python

[issue10329] trace.py and unicode in Python 3

Walter Dörwald added the comment: > STINNER Victor added the comment: > >> ... it complicates handling of the output of trace.py. >> For each file you have to do the encoding detection dance again ... > > What? You just have to call one function! tokenize.open() :-) Well, ok, > it's not com

[issue10329] trace.py and unicode in Python 3

STINNER Victor added the comment: > ... it complicates handling of the output of trace.py. > For each file you have to do the encoding detection dance again ... What? You just have to call one function! tokenize.open() :-) Well, ok, it's not commited yet, but it looks like most people agree:

[issue10329] trace.py and unicode in Python 3

Walter Dörwald added the comment: Using the original encoding of the Python source file might be the politically correct thing to do, but it complicates handling of the output of trace.py. For each file you have to do the encoding detection dance again. It would be great if I could specify wh

[issue10329] trace.py and unicode in Python 3

Alexander Belopolsky added the comment: Reopening as a reminder to add a unit test for this case. -- stage: needs patch -> unit test needed status: closed -> open ___ Python tracker ___

[issue10329] trace.py and unicode in Python 3

STINNER Victor added the comment: $ LANG=C ./python -m test.regrtest test_imp test_trace [1/2] test_imp [2/2] test_trace ... UnicodeEncodeError: 'ascii' codec can't encode character '\xa0' in position 5: ordinal not in range(128) issue10329.diff fixes this failure. The failure comes from a non

[issue10329] trace.py and unicode in Python 3

Antoine Pitrou added the comment: For the record, the test failure can reproduced by the following: $ LANG=C ./python -m test.regrtest test_imp test_trace [1/2] test_imp [2/2] test_trace /home/antoine/py3k/__svn__/Lib/unittest/case.py:402: ResourceWarning: unclosed file <_io.TextIOWrapper name

[issue10329] trace.py and unicode in Python 3

STINNER Victor added the comment: > 1. It opens the source file one more time. This is probably acceptable > because existing code already opens it at least four times when -m (show > missing) option is selected. (Twice in find_executable_linenos() and > twice in linecache.getlines(). Fixing

[issue10329] trace.py and unicode in Python 3

Alexander Belopolsky added the comment: Attached patch, issue10329.diff fixes the issue by setting the encoding of the coverage file to that of the source file. I am not 100% happy with this patch for the following reasons: 1. It opens the source file one more time. This is probably accepta

[issue10329] trace.py and unicode in Python 3

Alexander Belopolsky added the comment: On Fri, Nov 5, 2010 at 2:43 PM, Antoine Pitrou wrote: .. > Who are the consumers of the trace files? Is there a formal specification > or is Python the primary consumer? The trace files contain annotated python source code. There is no formal specificat

[issue10329] trace.py and unicode in Python 3

Antoine Pitrou added the comment: > I would be +0 on adding errors='replace' or 'backshlashreplace' to the > open() call in write_results_file(), but hardcoding encoding="utf-8" > is definitely not the right thing to do. Who are the consumers of the trace files? Is there a formal specificatio

[issue10329] trace.py and unicode in Python 3

Alexander Belopolsky added the comment: I don't think trace.diff is proposed for commit. I see it more as a supporting file for diagnosing the problem. I see two problems here: 1. Apparently OP's system opens files with encoding set to 'ascii' by default. This is not the case on any of the

[issue10329] trace.py and unicode in Python 3

Changes by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue10329] trace.py and unicode in Python 3

Changes by Antoine Pitrou : -- components: +Library (Lib) nosy: +belopolsky stage: -> patch review type: -> behavior versions: +Python 3.1, Python 3.2 ___ Python tracker ___ __

[issue10329] trace.py and unicode in Python 3

New submission from Walter Dörwald : It seems that on Python 3 (i.e. the py3k branch) trace.py can not handle source that includes Unicode characters. Running the test suite with code coverage info via ./python Lib/test/regrtest.py -T -N -uurlfetch,largefile,network,decimal sometimes fails