Alexander Belopolsky <belopol...@users.sourceforge.net> added the comment:
On Thu, Dec 2, 2010 at 11:49 PM, Éric Araujo <rep...@bugs.python.org> wrote: .. > Whether 1+2j is a literal or an expression is debatable. > I think +1 is an expression but 1+2j is a literal; neither should have a >space. With respect to implementation there is no debate: [TokenInfo(type=57 (ENCODING), string='utf-8', start=(0, 0), end=(0, 0), line=''), TokenInfo(type=2 (NUMBER), string='1', start=(1, 0), end=(1, 1), line='1+1j'), TokenInfo(type=53 (OP), string='+', start=(1, 1), end=(1, 2), line='1+1j'), TokenInfo(type=2 (NUMBER), string='1j', start=(1, 2), end=(1, 4), line='1+1j'), TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')] [TokenInfo(type=57 (ENCODING), string='utf-8', start=(0, 0), end=(0, 0), line=''), TokenInfo(type=53 (OP), string='-', start=(1, 0), end=(1, 1), line='-1'), TokenInfo(type=2 (NUMBER), string='1', start=(1, 1), end=(1, 2), line='-1'), TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')] (Who designed the tokenize interface, btw? I took me 3 tries to come up with the incantation above.) > > I’m not sure the language reference and the actual implementation are in > agreement here > (I have peephole optimizations in mind). Literals are atomic to the tokenizer. AST processes a stream of tokens. Peephole optimizations are irrelevant because these are hacks that operate on the bytecode when the lexical structure is all but forgotten. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue10562> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
