I'm trying to understand the output of the tokenize.generate_tokens()
generator. The token types returned seem to be more general than I'd
expect. For example, when fed the following line of code:
def func_a():
the (abbreviated) returned token tuples are as follows:
(NAME, def, ..., def func_a():)
(NAME , func_a, ..., def func_a():)
(OP, (, ..., def func_a():)
(OP, ), ..., def func_a():)
(OP, :, ..., def func_a():)
(NEWLINE, NEWLINE, ..., def func_a():)
It seems to me that the token '(' should be identified as 'LPAR' and
')' as 'RPAR', as found in the dictionary token.tok_name. What am I
missing here?
--
http://mail.python.org/mailman/listinfo/python-list
