New submission from Stéphane Wirtel:
For tokenize.py, 'def' is not a keyword but just a simple identifier.
>>> tokens = tokenize.generate_tokens(io.StringIO('async def foo():
>>> pass').readline)
>>> pprint.pprint(list(tokens))
[TokenInfo(type=55 (ASYNC), string='async', start=(1, 0), end=(1, 5),
line='async def foo(): pass'),
TokenInfo(type=1 (NAME), string='def', start=(1, 6), end=(1, 9), line='async
def foo(): pass'),
TokenInfo(type=1 (NAME), string='foo', start=(1, 10), end=(1, 13), line='async
def foo(): pass'),
TokenInfo(type=53 (OP), string='(', start=(1, 13), end=(1, 14), line='async
def foo(): pass'),
TokenInfo(type=53 (OP), string=')', start=(1, 14), end=(1, 15), line='async
def foo(): pass'),
TokenInfo(type=53 (OP), string=':', start=(1, 15), end=(1, 16), line='async
def foo(): pass'),
TokenInfo(type=1 (NAME), string='pass', start=(1, 17), end=(1, 21),
line='async def foo(): pass'),
TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]
>>> 'def' in keyword.kwlist
True
>>> tokens = tokenize.generate_tokens(io.StringIO('def foo(): pass').readline)
>>> pprint.pprint(list(tokens))
[TokenInfo(type=1 (NAME), string='def', start=(1, 0), end=(1, 3), line='def
foo(): pass'),
TokenInfo(type=1 (NAME), string='foo', start=(1, 4), end=(1, 7), line='def
foo(): pass'),
TokenInfo(type=53 (OP), string='(', start=(1, 7), end=(1, 8), line='def foo():
pass'),
TokenInfo(type=53 (OP), string=')', start=(1, 8), end=(1, 9), line='def foo():
pass'),
TokenInfo(type=53 (OP), string=':', start=(1, 9), end=(1, 10), line='def
foo(): pass'),
TokenInfo(type=1 (NAME), string='pass', start=(1, 11), end=(1, 15), line='def
foo(): pass'),
TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]
>>>
Is it normal ?
----------
messages: 254277
nosy: matrixise
priority: normal
severity: normal
status: open
title: def is not a keyword with tokenize.py
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25579>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
