Peter Ludemann <peter.ludem...@gmail.com> added the comment:
Looking at the suggested successor tools (redbaron, libCST, parso, awpa) ... all of them appear to use some variant of pgen2. But at some point Python will be using a PEG approach (PEP 617), and therefor the pgen2 approach apparently won't work. For a number of projects, it's important to have a parse tree that contains all the "whitespace" information (indent, dedent, comment, newline, etc.) As far as I can tell, the new PEG parser won't provide that, and it seems that none of the successor tools will be able to handle future versions of Python syntax. So, three questions: 1. Am I right that all proposed replacements (redbaron, libCST, parso, awpa) use some variation of the LL(1) and therefore will have trouble in the future? 2. Are there any plans (either part of the core development or as a project) for one of these replacements that is PEG-based? (Or a new project?) 3. Is Lib/ast.py going to continue being supported? (I infer that it will, with the change from LL(1) to PEG being mostly transparent - https://mail.python.org/archives/list/python-...@python.org/thread/HOZ2RI3FXUEMAT4XAX4UHFN4PKG5J5GR/#4D3B2NM2JMV2UKIT6EV5Q2A6XK2HXDEH ) If Lib/ast.py continues to be supported, I think I can see a way of providing functionality similar to lib2to3 (in terms of an AST-ish thing with "whitespace" from the source, sufficient for tools such as yapf, black, pykythe, pytype, mypy, etc.) as a kind of wrapper to ast.py. I suppose I should discuss this idea on python-dev? Is there an ongoing discussion? (I couldn't find any but might have been using the wrong search terms) ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue40360> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
