[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Benjamin Peterson added the comment: Applied in r70587. -- resolution: -> accepted status: open -> closed ___ Python tracker ___ ___

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

STINNER Victor added the comment: Oh, I see that setup.py uses warnings and so linecache is loaded by setup.py. -- ___ Python tracker ___ ___

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

STINNER Victor added the comment: @benjamin.peterson: The second version of my patch works correctly with the bootstraping. > I also think we should consider hard adding more modules > that are loaded at startup time to py3k already huge list. linecache is not loaded at startup time in py3k!

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

STINNER Victor added the comment: > I also think we should consider hard adding more modules > that are loaded at startup time to py3k already huge list. My patch uses tokenize modules in setup.py bootstrap. But it doesn't affect Python classic usage "python" or "python myscript.py". ___

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Changes by Antoine Pitrou : -- priority: -> normal stage: -> patch review versions: +Python 3.1 -Python 3.0 ___ Python tracker ___ __

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Changes by STINNER Victor : Removed file: http://bugs.python.org/file11678/linecache_refactor.patch ___ Python tracker ___ ___ Python-bugs-list

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

STINNER Victor added the comment: It was easy to remove the itertools dependency: only itertools.chain() was really used and a simple "for ...: yield" is enough to get the same behaviour (test_tokenize.py runs fine). With the new version of the patch, the bootstrap works correctly. Added fil

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Benjamin Peterson added the comment: I had to revert this because tokenize imports itertools. This is a problem when building (setup.py) because itertools doesn't exist, yet. I also think we should consider hard adding more modules that are loaded at startup time to py3k already huge list.

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Benjamin Peterson added the comment: Applied in r67713. -- nosy: +benjamin.peterson resolution: -> accepted status: open -> closed ___ Python tracker ___ ___

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

STINNER Victor added the comment: "This patch looks good." ok and then? ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: This patch looks good. -- nosy: +amaury.forgeotdarc ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Changes by STINNER Victor <[EMAIL PROTECTED]>: Removed file: http://bugs.python.org/file11679/tokenize_bom_utf8.patch ___ Python tracker <[EMAIL PROTECTED]> ___ ___

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

STINNER Victor <[EMAIL PROTECTED]> added the comment: I wrote a different (and better) patch for tokenize module: moved to the issue4021. ___ Python tracker <[EMAIL PROTECTED]> ___ ___

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

Changes by STINNER Victor <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file11679/tokenize_bom_utf8.patch ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

New submission from STINNER Victor <[EMAIL PROTECTED]>: linecache uses it own code to detect a Python script encoding whereas a function tokenize.detect_encoding() already exists. It does also convert bytes => unicode conversion of the file lines whereas open() already supports this feature.