[issue46289] AST: FomattedValue conversion's default value should be -1
Lysandros Nikolaou added the comment: I agree that A is probably the way to go. -- ___ Python tracker <https://bugs.python.org/issue46289> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47129] Improve errors messages in f-string syntax errors
Lysandros Nikolaou added the comment: I like the newest suggestion by @Jelle better as well. -- ___ Python tracker <https://bugs.python.org/issue47129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43748] Inconsistent grammar in new error message (introduced in 3.10.0a7)
Lysandros Nikolaou added the comment: We can change the wording to `cannot` in order for all of the error messages to be consistent. Would you like to propose a PR, Andre? -- ___ Python tracker <https://bugs.python.org/issue43748> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43798] Add position metadata to alias AST type
Lysandros Nikolaou added the comment: I'm okay with this, unless someone has any input on why the alias node should not have line/column info (and did not have it in the first place). -- nosy: +BTaskaya, gvanrossum, lys.nikolaou, pablogsal ___ Python tracker <https://bugs.python.org/issue43798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43798] Add position metadata to alias AST type
Lysandros Nikolaou added the comment: New changeset 75a06f067bd0a2687312e5f8e78f9075be76ad3a by Matthew Suozzo in branch 'master': bpo-43798: Add source location attributes to alias (GH-25324) https://github.com/python/cpython/commit/75a06f067bd0a2687312e5f8e78f9075be76ad3a -- ___ Python tracker <https://bugs.python.org/issue43798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43798] Add position metadata to alias AST type
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue43798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44122] let linter allow defining attribute outside __init__ if called in __init__
Lysandros Nikolaou added the comment: Hi Ohad and thanks for filing a report. This is not a problem with CPython and outside of our control. You'd have to ask this question to the PyCharm linter team (unfortunately I don't know where exactly you would be able to do that). Closing. -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue44122> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The first line (comment) of the parser.c is incorrect.
Lysandros Nikolaou added the comment: New changeset aef1b58dc8889e1bebaddf1e30f67b63a1b20f90 by Akira Nonaka in branch 'main': bpo-44345: Fix 'generated by' comment in parser.c (GH-26615) https://github.com/python/cpython/commit/aef1b58dc8889e1bebaddf1e30f67b63a1b20f90 -- nosy: +lys.nikolaou ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The first line (comment) of the parser.c is incorrect.
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44385] Some target-related rules are unreachable in the grammar
New submission from Lysandros Nikolaou : Three rules (targets, target and t_atom) in the grammar are unreachable. We were using them at some point in the past, but we never removed them when we replaced them with other rules. -- assignee: lys.nikolaou components: Parser messages: 395577 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Some target-related rules are unreachable in the grammar type: behavior versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44385> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44385] Some target-related rules are unreachable in the grammar
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +25241 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26655 ___ Python tracker <https://bugs.python.org/issue44385> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44385] Some target-related rules are unreachable in the grammar
Change by Lysandros Nikolaou : -- pull_requests: +25245 pull_request: https://github.com/python/cpython/pull/26659 ___ Python tracker <https://bugs.python.org/issue44385> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44385] Some target-related rules are unreachable in the grammar
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue44385> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44838] SyntaxError: New message "expected 'else' after 'if' expression" wrongly shown
Lysandros Nikolaou added the comment:
Oh, this is a difficult one.
It's caused by GH-27506 that was recently added to produce better errors for
things like `print('hello' if something)`, where the correct syntax is
`print('hello' if something else 'hi').
This is going to have a lot of false positives though, especially in cases like
this, where an if statement immediately follows a line with unclosed
parentheses.
What do you think, Pablo? Should we maybe revert this?
--
___
Python tracker
<https://bugs.python.org/issue44838>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39031] Inconsistent lineno and col_offset info when parsing elif
New submission from Lysandros Nikolaou : While working on pegen, we came across an inconsistency on how line number and column offset info is stored for (el)if nodes. When parsing a very simple if-elif construct like if a: pass elif b: pass the following parse tree gets generated: Module( body=[ If( test=Name(id="a", ctx=Load(), lineno=1, col_offset=3, end_lineno=1, end_col_offset=4), body=[Pass(lineno=2, col_offset=4, end_lineno=2, end_col_offset=8)], orelse=[ If( test=Name( id="b", ctx=Load(), lineno=3, col_offset=5, end_lineno=3, end_col_offset=6 ), body=[Pass(lineno=4, col_offset=4, end_lineno=4, end_col_offset=8)], orelse=[], lineno=3, col_offset=5, end_lineno=4, end_col_offset=8, ) ], lineno=1, col_offset=0, end_lineno=4, end_col_offset=8, ) ], type_ignores=[], ) There is the inconsistency that the column offset for the if statement is 0, thus the if statement starts with the keyword if, whereas the column offset for elif if 5, which means that the elif keyword is skipped. As Guido suggests over at https://github.com/gvanrossum/pegen/issues/107#issuecomment-565135047 we could very easily change Python/ast.c so that the elif statement start with the elif keyword as well. I have a PR ready! -- messages: 358304 nosy: lys.nikolaou priority: normal severity: normal status: open title: Inconsistent lineno and col_offset info when parsing elif type: behavior versions: Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39031> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39031] Inconsistency with lineno and col_offset info when parsing elif
Change by Lysandros Nikolaou : -- title: Inconsistent lineno and col_offset info when parsing elif -> Inconsistency with lineno and col_offset info when parsing elif ___ Python tracker <https://bugs.python.org/issue39031> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39031] Inconsistency with lineno and col_offset info when parsing elif
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +17056 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17582 ___ Python tracker <https://bugs.python.org/issue39031> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39031] Inconsistency with lineno and col_offset info when parsing elif
Change by Lysandros Nikolaou : -- pull_requests: +17059 pull_request: https://github.com/python/cpython/pull/17585 ___ Python tracker <https://bugs.python.org/issue39031> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39031] Inconsistency with lineno and col_offset info when parsing elif
Lysandros Nikolaou added the comment: There was a bug in my last PR, hopefully I will get a fix some time later today. The bug is as follows: I only updated the asdl_seq_SET call for an elif without an else, if an else is included then the behavior is like before. After my last PR it looks like this, parsing 2:if a: 3:pass 4:elif b: 5:pass outputs the following AST: Module( body=[ If( test=Name(id="a", ctx=Load(), lineno=2, col_offset=3, end_lineno=2, end_col_offset=4), body=[Pass(lineno=3, col_offset=4, end_lineno=3, end_col_offset=8)], orelse=[ If( test=Name( id="b", ctx=Load(), lineno=4, col_offset=5, end_lineno=4, end_col_offset=6 ), body=[Pass(lineno=5, col_offset=4, end_lineno=5, end_col_offset=8)], orelse=[], lineno=4, col_offset=0, end_lineno=5, end_col_offset=8, ) ], lineno=2, col_offset=0, end_lineno=5, end_col_offset=8, ) ], type_ignores=[], ) On the other hand parsing 2:if a: 3:pass 4:elif b: 5:pass 6:else: 7:pass outputs Module( body=[ If( test=Name( id="a", ctx=Load(), lineno=2, col_offset=3, end_lineno=2, end_col_offset=4, ), body=[Pass(lineno=3, col_offset=4, end_lineno=3, end_col_offset=8)], orelse=[ If( test=Name( id="b", ctx=Load(), lineno=4, col_offset=5, end_lineno=4, end_col_offset=6, ), body=[Pass(lineno=5, col_offset=4, end_lineno=5, end_col_offset=8)], orelse=[ Pass(lineno=7, col_offset=4, end_lineno=7, end_col_offset=8) ], lineno=4, col_offset=5, end_lineno=7, end_col_offset=8, ) ], lineno=2, col_offset=0, end_lineno=7, end_col_offset=8, ) ], type_ignores=[], ) -- ___ Python tracker <https://bugs.python.org/issue39031> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39031] Inconsistency with lineno and col_offset info when parsing elif
Change by Lysandros Nikolaou : -- pull_requests: +17068 pull_request: https://github.com/python/cpython/pull/17596 ___ Python tracker <https://bugs.python.org/issue39031> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39080] Inconsistency with Starred Expression line/col info
New submission from Lysandros Nikolaou : When a starred expression like *[0, 1] is parsed, the following AST gets generated: Module( body=[ Expr( value=Starred( value=List( elts=[ Constant( value=0, kind=None, lineno=1, col_offset=2, end_lineno=1, end_col_offset=3, ), Constant( value=1, kind=None, lineno=1, col_offset=5, end_lineno=1, end_col_offset=6, ), ], ctx=Load(), lineno=1, col_offset=1, end_lineno=1, end_col_offset=7, ), ctx=Load(), lineno=1, col_offset=0, end_lineno=1, end_col_offset=7, ), lineno=1, col_offset=0, end_lineno=1, end_col_offset=7, ) ], type_ignores=[], ) But, when a starred expression is an argument to a function call then the line/col info are wrong (end_col_offset is always equal to col_offset + 1): Module( body=[ Expr( value=Call( func=Name( id="f", ctx=Load(), lineno=1, col_offset=0, end_lineno=1, end_col_offset=1 ), args=[ Starred( value=List( elts=[ Constant( value=0, kind=None, lineno=1, col_offset=4, end_lineno=1, end_col_offset=5, ), Constant( value=1, kind=None, lineno=1, col_offset=7, end_lineno=1, end_col_offset=8, ), ], ctx=Load(), lineno=1, col_offset=3, end_lineno=1, end_col_offset=9, ), ctx=Load(), lineno=1, col_offset=2, end_lineno=1, end_col_offset=9, ) ], keywords=[], lineno=1, col_offset=0, end_lineno=1, end_col_offset=10, ), lineno=1, col_offset=0, end_lineno=1, end_col_offset=10, ) ], type_ignores=[], ) -- components: Interpreter Core messages: 358584 nosy: lys.nikolaou priority: normal severity: normal status: open title: Inconsistency with Starred Expression line/col info type: behavior versions: Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39080> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39080] Inconsistency with Starred Expression line/col info
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +17113 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17645 ___ Python tracker <https://bugs.python.org/issue39080> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39235] Generator expression has wrong line/col info when inside a Call object
New submission from Lysandros Nikolaou : A normal generator expression like (i for i in a) produces the following AST: Module( body=[ Expr( value=GeneratorExp( elt=Name( id="i", ctx=Load(), lineno=1, col_offset=1, end_lineno=1, end_col_offset=2 ), generators=[ comprehension( target=Name( id="i", ctx=Store(), lineno=1, col_offset=7, end_lineno=1, end_col_offset=8, ), iter=Name( id="a", ctx=Load(), lineno=1, col_offset=12, end_lineno=1, end_col_offset=13, ), ifs=[], is_async=0, ) ], lineno=1, *col_offset=0,* end_lineno=1, *end_col_offset=14,* ), lineno=1, col_offset=0, end_lineno=1, end_col_offset=14, ) ], type_ignores=[], ) But when calling a function with a generator expression as an argument, something is off: Module( body=[ Expr( value=Call( func=Name( id="f", ctx=Load(), lineno=1, col_offset=0, end_lineno=1, end_col_offset=1 ), args=[ GeneratorExp( elt=Name( id="i", ctx=Load(), lineno=1, col_offset=2, end_lineno=1, end_col_offset=3, ), generators=[ comprehension( target=Name( id="i", ctx=Store(), lineno=1, col_offset=8, end_lineno=1, end_col_offset=9, ), iter=Name( id="a", ctx=Load(), lineno=1, col_offset=13, end_lineno=1, end_col_offset=14, ), ifs=[], is_async=0, ) ], lineno=1, *col_offset=1,* end_lineno=1, *end_col_offset=2,* ) ], keywords=[], lineno=1, col_offset=0, end_lineno=1, end_col_offset=15, ), lineno=1, col_offset=0, end_lineno=1, end_col_offset=15, ) ], type_ignores=[], ) I'm not sure if this is intentional or not, because there is a call to copy_location in Python/ast.c:3149. If this call to copy_location is removed, the inconsistency goes away. -- components: Interpreter Core messages: 359454 nosy: lys.nikolaou priority: normal severity: normal status: open title: Generator expression has wrong line/col info when inside a Call object type: behavior versions: Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39235] Generator expression has wrong line/col info when inside a Call object
Lysandros Nikolaou added the comment: Do you think it is okay to just remove the call to copy_location? -- ___ Python tracker <https://bugs.python.org/issue39235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16806] col_offset is -1 and lineno is wrong for multiline string expressions
Change by Lysandros Nikolaou : -- pull_requests: +17320 pull_request: https://github.com/python/cpython/pull/17582 ___ Python tracker <https://bugs.python.org/issue16806> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39564] Parsed expression has wrong col_offset when concatenating f-strings
Change by Lysandros Nikolaou : -- title: Parsed expression has wrong line/col info when concatenating f-strings -> Parsed expression has wrong col_offset when concatenating f-strings ___ Python tracker <https://bugs.python.org/issue39564> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39564] Parsed expression has wrong line/col info when concatenating f-strings
New submission from Lysandros Nikolaou :
When concatenating f-strings, if there is an expression in any STRING node
other than the first, col_offset of the parsed expression has a wrong value.
For example, parsing f"hello" f"{world}" outputs the following AST:
Module(
body=[
Expr(
value=JoinedStr(
values=[
Constant(
value="hello",
kind=None,
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=19,
),
FormattedValue(
value=Name(
id="world",
ctx=Load(),
lineno=1,
*col_offset=1,*
end_lineno=1,
*end_col_offset=6,*
),
conversion=-1,
format_spec=None,
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=19,
),
],
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=19,
),
lineno=1,
col_offset=0,
end_lineno=1,
end_col_offset=19,
)
],
type_ignores=[],
)
Here, col_offset and end_col_offset are wrong in the parsed NAME 'world'.
--
components: Interpreter Core
messages: 361456
nosy: gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Parsed expression has wrong line/col info when concatenating f-strings
versions: Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue39564>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39564] Parsed expression has wrong col_offset when concatenating f-strings
Lysandros Nikolaou added the comment: Also note that for pegen this is a solved problem. If we decide to change the parser, it may not be worth it to spend too much time on a universal solution. -- ___ Python tracker <https://bugs.python.org/issue39564> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39564] Parsed expression has wrong col_offset when concatenating f-strings
Lysandros Nikolaou added the comment: > Can I ask how pegen solved this problem? Did you move parsing of f-strings > into the grammar? No, we actually do a very similar thing to what ast.c does. I think there is only one major difference between what pegen does and what ast.c does. If I understand correctly, fstring_compile_expr calls fstring_fix_node_location with parent set to the whole STRING+ node. We OTOH only pass as parent the current STRING token (we work with tokens returned by the tokenizer) and then walk the AST tree and adjust the line/col_offset based on that. I'm not sure that this is bug-free, but after some testing, I could find any obvious errors. If you want, you can take a loot at the code on https://github.com/gvanrossum/pegen/pull/183. -- ___ Python tracker <https://bugs.python.org/issue39564> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39579] Attribute node in a decorator has wrong end_col_offset
New submission from Lysandros Nikolaou : There is a problem with the end_col_offset of nested Attribute nodes in decorators. For example, parsing @a.b.c def f(): pass produces the following AST tree (part): decorator_list=[ Attribute( value=Attribute( value=Name( id="a", ctx=Load(), lineno=1, col_offset=1, end_lineno=1, end_col_offset=2, ), attr="b", ctx=Load(), lineno=1, col_offset=1, end_lineno=1, *end_col_offset=6*, ), attr="c", ctx=Load(), lineno=1, col_offset=1, end_lineno=1, end_col_offset=6, ) ], Note that the Attribute node with attr="b" has end_col_offset=6, while it should actually be 4. -- components: Interpreter Core messages: 361595 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Attribute node in a decorator has wrong end_col_offset versions: Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39579] Attribute node in a decorator has wrong end_col_offset
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +17781 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18405 ___ Python tracker <https://bugs.python.org/issue39579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39579] Attribute node in a decorator has wrong end_col_offset
Change by Lysandros Nikolaou : -- pull_requests: +17783 pull_request: https://github.com/python/cpython/pull/18408 ___ Python tracker <https://bugs.python.org/issue39579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39474] col_offset for parenthesized expressions looks weird
Change by Lysandros Nikolaou : -- nosy: +lys.nikolaou ___ Python tracker <https://bugs.python.org/issue39474> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39474] col_offset for parenthesized expressions looks weird
Lysandros Nikolaou added the comment: Are you going to work on a patch then, Batuhan? -- ___ Python tracker <https://bugs.python.org/issue39474> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39633] venv does not include python. symlink by default
New submission from Lysandros Nikolaou : At the moment running python -m venv venv or python3 -m venv venv creates a virtual environment that does not contain a python. symlink, which results in executing whatever the default python is when running i.e. python. within an activated virtual env. OTOH if one runs python. -m venv venv, then everything is OK. Would it be possible to include a python. symlink in all cases? If not, then I think we should update the docs to mention that somewhere, since it took me quite a while to figure this out. -- components: Library (Lib) messages: 361993 nosy: lys.nikolaou priority: normal severity: normal status: open title: venv does not include python. symlink by default versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue39633> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39633] venv does not include python. symlink by default
Lysandros Nikolaou added the comment: The thing is that some projects I've worked on, like pegen, include a Makefile which uses `pythonX.Y` to choose the version and I have to change it to `python` quite frequently. I can't really figure out any way this could harm anyone, other than what you said (running `pythonX.Y` with the wrong values for X and Y), but even in that case you get something like `command not fonund` and that's an easy enough typo to find. -- ___ Python tracker <https://bugs.python.org/issue39633> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39633] venv does not include python. symlink by default
Lysandros Nikolaou added the comment: You're right! That could be very confusing. But isn't this risk already there, since this could happen if one runs `python3.8 -m venv venv` and then `python3.7` with the venv activated? -- ___ Python tracker <https://bugs.python.org/issue39633> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39633] venv does not include python. symlink by default
Lysandros Nikolaou added the comment: That's right. I'll do that then. Thanks for baring with me. Should we update the docs to mentiom this or just close this issue? On Sat, Feb 15, 2020, 22:33 Vinay Sajip wrote: > > Vinay Sajip added the comment: > > In general, people should be working in venvs. That means that libraries > should assume they are being installed into venvs. Therefore makefiles > should be invoking plain "python" rather than "pythonX.Y", as they can > never be sure which exact interpreter they're running under (you might have > a newer one than the one in the Makefile - that shouldn't fail). Why not > raise this as an issue with these projects? > > -- > > ___ > Python tracker > <https://bugs.python.org/issue39633> > ___ > -- ___ Python tracker <https://bugs.python.org/issue39633> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
New submission from Lysandros Nikolaou :
While testing pegen, we found this out:
Python 3.9.0a5+ (heads/pegen:502dfb719e, Apr 10 2020, 20:57:05)
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> fur''
File "", line 1
fur''
^
SyntaxError: invalid syntax
>>> eval("fur''")
Traceback (most recent call last):
File "", line 1, in
File "", line 1
fur''
^
SyntaxError: unexpected EOF while parsing
--
components: Interpreter Core
messages: 366143
nosy: lys.nikolaou
priority: normal
severity: normal
status: open
title: Different error messages for same error - invalid string prefixes
type: behavior
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue40246>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Change by Lysandros Nikolaou : -- nosy: +gvanrossum, pablogsal ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: After some investigating, I found out that this is happening because when we execute `fur''` in the interactive interpreter there is an implicit newline, which means that EOF does not get reached. OTOH there is no newline when passing it as a string inside an `eval` call. After the tokenizer encounters the first quote, it calls `tok_nextc` twice more, in order to differentiate between a single-quoted and a triple-quoted string, thus reaching EOF and proapgating an EOF error up through the `perrdetail` struct. -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: At the moment, if the prefix is invalid, it gets tokenized as a NAME node and the string following is tokenized as a STRING node with no prefix. ╰─ echo "ur''" | ./python -m tokenize 1,0-1,2:NAME 'ur' 1,2-1,4:STRING "''" 1,4-1,5:NEWLINE'\n' 2,0-2,0:ENDMARKER '' ╰─ echo "f''" | ./python -m tokenize 1,0-1,3:STRING "f''" 1,3-1,4:NEWLINE'\n' 2,0-2,0:ENDMARKER '' -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: In my understanding of the C code that's what the C tokenizer is doing as well. Here's the relevant snippet of the tokenizer (https://github.com/python/cpython/blob/4b222c9491d1700e9bdd98e6889b8d0ea1c7321e/Parser/tokenizer.c#L1358): When the tokenizer sees a valid identifier start, it goes into a loop that checks for a valid combination of string prefixes. If the combination is valid and it sees a quote directly after that, it goto's to the STRING-handling code. If not, then it breaks out of the loop and returns a NAME node. Am I missing something? -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: I have working code that checks if there is a quotation mark right after an identifier. Here is an example: ╰─ ./python Python 3.9.0a5+ (heads/pegen-dirty:502dfb719e, Apr 11 2020, 14:43:12) [GCC 9.2.1 20191008] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ur'' File "", line 1 ur'' ^ SyntaxError: invalid string prefix One observation about this is that it has precedence over an EOL error: >>> ur' File "", line 1 ur' ^ SyntaxError: invalid string prefix Would that be acceptable? -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +18830 stage: -> patch review pull_request: https://github.com/python/cpython/pull/19476 ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40267] Error message differs when an expression is in an fstring
New submission from Lysandros Nikolaou :
There are cases, where the error message differs, when an expression is being
parsed inside an fstring. For example:
>>> f'{x+}'
File "", line 1
(x+)
^
SyntaxError: unexpected EOF while parsing
>>> (x+)
File "", line 1
(x+)
^
SyntaxError: invalid syntax
Or with lambda definitions:
>>> f'{lambda x:x}'
File "", line 1
(lambda x)
^
SyntaxError: unexpected EOF while parsing
>>> (lambda x)
File "", line 1
(lambda x)
^
SyntaxError: invalid syntax
--
components: Interpreter Core
messages: 366272
nosy: gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Error message differs when an expression is in an fstring
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue40267>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40267] Error message differs when an expression is in an fstring
Lysandros Nikolaou added the comment:
It seems that this is actually a bit bigger than this and it is not specific to
f-strings.
The error message *always* changes to `unexpected EOF while parsing` if there
is an error with the last character of the input and no newline follows. For
example, as made clear to me by Guido, there are even differences in error
messages between exec'ing and eval'ing something:
>>> exec('x+')
Traceback (most recent call last):
File "", line 1, in
File "", line 1
x+
^
SyntaxError: invalid syntax
>>> eval('x+')
Traceback (most recent call last):
File "", line 1, in
File "", line 1
x+
^
SyntaxError: unexpected EOF while parsing
That's because the tokenizer adds an implicit newline to the input string,
before tokenizing it, when the input comes from an exec call. (see
https://github.com/python/cpython/blob/14d5331eb5e6c38be12bad421bd59ad0fac9e448/Parser/tokenizer.c#L648)
And that's not limited to a character missing, as suggested by the error
message. Even when the last character itself generates a SyntaxError, the error
message remains "unexpcted EOF while parsing":
>>> x+@
File "", line 1
x+@
^
SyntaxError: invalid syntax
>>> eval('x+@')
Traceback (most recent call last):
File "", line 1, in
File "", line 1
x+@
^
SyntaxError: unexpected EOF while parsing
Thus, a very simple fix to the specific fstring problem of this issue would be
to add a newline to the string that gets parsed to the parser in
fstring_compile_expr in ast.c, but I guess it'd be better to fix the tokenizer
itself, if this is considered a bug.
--
___
Python tracker
<https://bugs.python.org/issue40267>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40267] Error message differs when an expression is in an fstring
Change by Lysandros Nikolaou : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue40267> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40267] Error message differs when an expression is in an fstring
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +18871 stage: -> patch review pull_request: https://github.com/python/cpython/pull/19521 ___ Python tracker <https://bugs.python.org/issue40267> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- nosy: +lys.nikolaou ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +18991 pull_request: https://github.com/python/cpython/pull/19669 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Lysandros Nikolaou added the comment: > test_peg_generator emits compiler warnings These are expected. Is this a problem? -- ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40370] AIX: ld_so_aix not found during test of test_peg_generator
Lysandros Nikolaou added the comment: I propose skipping test_peg_generator for now, so that we don't hold the alpha6 release. It actually only tests the generator and not the parser and since we don't plan on adding new operators in the very near future, we can skip it and fix it after the release. -- ___ Python tracker <https://bugs.python.org/issue40370> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +18999 pull_request: https://github.com/python/cpython/pull/19675 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Lysandros Nikolaou added the comment: > Warning -- files was modified by test_peg_generator > Before: [] > After: ['parse_d.cp39-win_amd64.pdb', 'vc140.pdb'] Currently working on this. -- ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19014 pull_request: https://github.com/python/cpython/pull/19694 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Lysandros Nikolaou added the comment: > test_peg_generator leaks references. This is now fixed. -- ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19065 pull_request: https://github.com/python/cpython/pull/19743 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19066 pull_request: https://github.com/python/cpython/pull/19744 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19091 pull_request: https://github.com/python/cpython/pull/19771 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19094 pull_request: https://github.com/python/cpython/pull/19774 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19098 pull_request: https://github.com/python/cpython/pull/19778 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: On bpo-40334 @vstinner reported that this change broke some code in turtledemo. The code looks like this: print(dict(state=clear, bg="#d00" if clear else"#fca")) Due to the absence of a space between `else` and `"`, the else keyword is now interpreted as an invalid string prefix. Is that acceptable? -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: A possible solution would be to only emit a SyntaxError if the NAME directly preceding a STRING token contains one of the valid string prefixes (either one of 'f', 'r', 'u', 'b'). This would still output a nicer error message, but would not break code like the one of the example. What do you think about this? -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: What's also possible is to handle keywords at tokenizer level and return a different token type for each one of them (like with async and await), which is currently handled at parser level. This would enable us to allow reserved keywords right before a STRING token, thus covering all the possible broken code cases, but continue disallowing arbitrary NAMEs, which would mean better error-reporting in the case of invalid prefixes. -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19147 pull_request: https://github.com/python/cpython/pull/19827 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19157 pull_request: https://github.com/python/cpython/pull/19839 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40449] multi-line f-string, syntaxerror points to wrong line
Lysandros Nikolaou added the comment:
This has been fixed in 3.9.0a6 due to the new parser:
╰─ cat a.py
s = ("apricot "
"pineapple"
f"shallot{")
╰─ ./python.exe a.py
File "/Users/lysnikolaou/Repositories/cpython/a.py", line 3
f"shallot{")
^
SyntaxError: f-string: expecting '}'
--
nosy: +lys.nikolaou
___
Python tracker
<https://bugs.python.org/issue40449>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40449] multi-line f-string, syntaxerror points to wrong line
Lysandros Nikolaou added the comment: Since this issue is now resolved, I'm closing it, after I consulted with pablogsal. -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40449> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19179 pull_request: https://github.com/python/cpython/pull/19865 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- type: resource usage -> enhancement ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Lysandros Nikolaou added the comment: This isn't related to this issue, but to bpo-40246. Please report it there as well. -- ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19197 pull_request: https://github.com/python/cpython/pull/19887 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Change by Lysandros Nikolaou : -- pull_requests: +19199 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/19888 ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Change by Lysandros Nikolaou : -- nosy: +lys.nikolaou nosy_count: 3.0 -> 4.0 pull_requests: +19198 pull_request: https://github.com/python/cpython/pull/19888 ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Change by Lysandros Nikolaou : -- pull_requests: -19198 ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Change by Lysandros Nikolaou : -- pull_requests: +19200 pull_request: https://github.com/python/cpython/pull/19888 ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Change by Lysandros Nikolaou : -- pull_requests: -19200 ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Lysandros Nikolaou added the comment: I opened a pull request with an invalid bpo issue number so I unlinked it. Then a completely unrelated PR was is shown as linked by me, but I didn't do that. Unlinking that as well. -- ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Change by Lysandros Nikolaou : -- pull_requests: +19201 pull_request: https://github.com/python/cpython/pull/19888 ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Lysandros Nikolaou added the comment: PRs keep getting linked to this issue under my name, without me doing anything. -- ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Lysandros Nikolaou added the comment: After Serhiy unlinked the linked PR, it did not happen again, so it may have been an one-off hiccup. I updated the description GH-19888 nevertheless. -- ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: The revert is in. Now the question is if we want to take additional action to address the original issue of this. -- ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40426] Unable to use lowercase hexadecimal digits for percent encoding
Change by Lysandros Nikolaou : -- nosy: -lys.nikolaou ___ Python tracker <https://bugs.python.org/issue40426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Lysandros Nikolaou added the comment: Ok, I'm closing this, after consulting with Guido. -- resolution: -> wont fix stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19278 pull_request: https://github.com/python/cpython/pull/19962 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19279 pull_request: https://github.com/python/cpython/pull/19963 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19290 pull_request: https://github.com/python/cpython/pull/19973 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between pegen and traceback SyntaxErrors
New submission from Lysandros Nikolaou : Since traceback is programmed to match the SyntaxErrors emitted by the old parser, there are some inconsistencies between how it formats SyntaxErrors and how the new parser does it. One example for such behaviour is the following: New Parser: ╰─ ./python.exe Python 3.9.0a6+ (heads/master:4638c64295, May 7 2020, 16:47:53) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> x = 1 | 2 | File "", line 1 x = 1 | 2 | ^ SyntaxError: invalid syntax Old parser and traceback module: ╰─ ./python.exe -X oldparser Python 3.9.0a6+ (heads/master:4638c64295, May 7 2020, 16:47:53) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> x = 1 | 2 | File "", line 1 x = 1 | 2 | ^ SyntaxError: invalid syntax We actually think that the new parser does a better job here, since there is nothing wrong with the `|` itself, the error is that a newline follows it and that's where the caret should point to. Tests that are testing this are currently skipped. I think we should change traceback to match the new parser, wherever we think the new parser does a better job, and "unskip" these tests. -- components: Library (Lib) messages: 368353 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Inconsistencies between pegen and traceback SyntaxErrors type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue40546> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between PEG parser and traceback SyntaxErrors
Change by Lysandros Nikolaou : -- title: Inconsistencies between pegen and traceback SyntaxErrors -> Inconsistencies between PEG parser and traceback SyntaxErrors ___ Python tracker <https://bugs.python.org/issue40546> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between PEG parser and traceback SyntaxErrors
Lysandros Nikolaou added the comment:
Guido:
> I don't understand why the traceback module is implicated.
The traceback module is implicated, because it currently does not allow the
caret to point to a position after the error line.
In format_exception_only there is the following snippet of code:
if offset is not None:
caretspace = badline.rstrip('\n')
offset = min(len(caretspace), offset) - 1
caretspace = caretspace[:offset].lstrip()
# non-space whitespace (likes tabs) must be kept for alignment
caretspace = ((c.isspace() and c or ' ') for c in caretspace)
yield '{}^\n'.format(''.join(caretspace))
There are two things here, that put together contradict the way that pegen
reports errors. `badline` gets rstripped and then the offset is changed to
point to the end of the string, in case it is currently pointing after its end.
That basically does not allow SyntaxErrors to be formatted the way they
currently are, when using the new parser.
For example:
╰─ ./python.exe
Python 3.9.0a6+ (heads/master:4638c64295, May 7 2020, 16:47:53)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> x = 1 | 2 |
File "", line 1
x = 1 | 2 |
^
SyntaxError: invalid syntax
But:
╰─ ./python.exe
Python 3.9.0a6+ (heads/master:4638c64295, May 7 2020, 16:47:53)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import traceback
>>> try:
... exec('x = 1 | 2 |')
... except SyntaxError as se:
... exc = se
...
>>> traceback.print_exception(type(exc), exc, exc.__traceback__)
Traceback (most recent call last):
File "", line 2, in
File "", line 1
x = 1 | 2 |
^
SyntaxError: invalid syntax
I might be missing something here, but I think that that's the traceback
module's "fault".
--
___
Python tracker
<https://bugs.python.org/issue40546>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between PEG parser and traceback SyntaxErrors
Lysandros Nikolaou added the comment: > But if the col_offset points way past the text, what should happen? The > clipping there seems reasonable. Note that if a file is being parsed and not a string the behaviour is this: ╰─ cat -e t.py x = 1 | 2 |$ ╰─ ./python.exe t.py File "/Users/lysnikolaou/repos/cpython/t.py", line 1 x = 1 | 2 | ^ SyntaxError: invalid syntax ╰─ ./python.exe -X oldparser t.py File "/Users/lysnikolaou/repos/cpython/t.py", line 1 x = 1 | 2 | ^ SyntaxError: invalid syntax Would we want to also change this? If not, then I guess that we should as closely as possible match this behaviour with strings as well, which means not clipping. -- ___ Python tracker <https://bugs.python.org/issue40546> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40555] pegen (PEP 617): bug in error handling
Lysandros Nikolaou added the comment: That's because we're not checking for errors after a loop is done. Pushing a fix. -- ___ Python tracker <https://bugs.python.org/issue40555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between PEG parser and traceback SyntaxErrors
Lysandros Nikolaou added the comment: > Hm, but there the old parser seems at fault: it points to the last space > rather than beyond it? Both parsers seem to have the same behaviour, when parsing files. I just found out that even the new parser points to the `|` if there is not trailing whitespace: ╰─ cat -e t.py x = 1 | 2 |$ ╰─ ./python.exe t.py File "/Users/lysnikolaou/Repositories/cpython/t.py", line 1 x = 1 | 2 | ^ SyntaxError: invalid syntax ╰─ ./python.exe -X oldparser t.py File "/Users/lysnikolaou/Repositories/cpython/t.py", line 1 x = 1 | 2 | ^ SyntaxError: invalid syntax I'll investigate more tomorrow. -- ___ Python tracker <https://bugs.python.org/issue40546> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19315 pull_request: https://github.com/python/cpython/pull/20003 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between PEG parser and traceback SyntaxErrors
Lysandros Nikolaou added the comment: Guido: > I wonder if this is the fix we're looking for? I think this is the most sensible thing to do, yes. I guess, we also have to change both parsers to match this behaviour, right? -- ___ Python tracker <https://bugs.python.org/issue40546> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40599] Improve error messages with expected keywords
Lysandros Nikolaou added the comment: I also concur with Guido here. I have played around with other languages and I dislike getting a long list of expected tokens, that are not helpful, if not actually confusing sometimes. I think that the current generic SyntaxError description together with the error caret actually does a good job of directing someone to close where the error is, without providing too much information that might be misleading. -- ___ Python tracker <https://bugs.python.org/issue40599> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19359 pull_request: https://github.com/python/cpython/pull/20050 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40618] PEG Parser: Invalid targets for augassign and except succeed
New submission from Lysandros Nikolaou : Tuples and lists, which are invalid augmented assignment targets, currently get successfully parsed, without failing. ➜ cpython git:(targets-fix) ✗ ./python -X oldparser Python 3.9.0a6+ (heads/master-dirty:75cd8e48c6, May 14 2020, 03:01:42) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> (a, b) += (1, 2) File "", line 1 SyntaxError: illegal expression for augmented assignment ➜ cpython git:(pr/20076) ./python Python 3.9.0a6+ (heads/pr/20076:bcf5be2772, May 14 2020, 02:15:37) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> (a, b) += 1 SystemError: invalid node type (26) for augmented assignment We also accept arbitrary targets in `except` clauses, while we should only be accepting NAME nodes: ➜ cpython git:(master) ./python Python 3.9.0a6+ (heads/master:75cd8e48c6, May 14 2020, 03:14:03) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> try: ... pass ... except Exception as a.b: ... pass ... [1]135956 segmentation fault (core dumped) ./python -- assignee: lys.nikolaou components: Interpreter Core messages: 368809 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: PEG Parser: Invalid targets for augassign and except succeed type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue40618> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40618] PEG Parser: Invalid targets for augassign and except succeed
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +19387 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20083 ___ Python tracker <https://bugs.python.org/issue40618> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
