Jim <cook_...@yahoo.com> wrote:
>I'm trying to understand the output of the tokenize.generate_tokens()
>generator. The token types returned seem to be more general than I'd
>expect. For example, when fed the following line of code:
>
>def func_a():
>...
>It seems to me that the token '(' should be identified as 'LPAR' and
>')' as 'RPAR', as found in the dictionary token.tok_name. What am I
>missing here?
Did you read the module? Right at the top, it says:
<quote>
It is designed to match the working of the Python tokenizer exactly, except
that it produces COMMENT tokens for comments and gives type OP for all
operators
</quote>
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list
