Bugs item #1184112, was opened at 2005-04-15 21:55
Message generated for change (Comment added) made by tjreedy
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184112&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Eric Huss (ehuss)
Assigned to: Nobody/Anonymous (nobody)
Summary: Missing trailing newline with comment raises SyntaxError
Initial Comment:
The following illustrates a problem with the parser
handling the lack of trailing newlines:
>>> parser.suite('def foo():\n\tpass\n\n# comment')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<string>", line 4
# comment
^
SyntaxError: invalid syntax
>>> parser.suite('def foo():\n\tpass\n\n# comment\n')
<parser.st object at 0x847f0a0>
This is similar to bug 501622, however, this only seems
to happen when you have an indented block, followed by
a comment line that has no trailing newline.
I traced through tokenizer.c and whittled down the issue
into tok_get(). In the statement where it is processing
the comment character and looking at the tabforms, in
the first case this will end up with 'c' equal to EOF
whereas in the second case "c" will eqaul '\n'. When it
equals EOF, it is unable to do the cleanup necessary to
emit the DEDENT token (it immediately bails out with
ENDMARKER which causes parsetok() to barf because
the indentation level is still 1 inside tok_state).
Attached is a patch of a little hack I made that seems
to fix the problem. Although it seems to be a safe thing
to do, it is definitely a hack.
----------------------------------------------------------------------
>Comment By: Terry J. Reedy (tjreedy)
Date: 2006-03-17 11:48
Message:
Logged In: YES
user_id=593130
As noted by F.Lundh on pydev list,
http://docs.python.org/lib/built-in-funcs.html
says "When compiling multi-line statements, two caveats
apply: [...] and the input must be terminated by at least
one newline character" so it appears that doc == behavior.
Should this be closed? or both changed?
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2006-03-17 08:51
Message:
Logged In: YES
user_id=1038590
Confirmed on SVN HEAD using:
exec """
def foo():
pass
#comment"""
(Blows up with a syntax error)
----------------------------------------------------------------------
Comment By: Eric Huss (ehuss)
Date: 2005-04-15 21:57
Message:
Logged In: YES
user_id=393416
Well, wonderful sourceforge is barfing with the
error "ArtifactFile: Could not open file for writing" when trying
to upload my patch, so I'll just post it in the comment here.
Very sorry. :(
--- tokenizer.c 3 Feb 2004 22:53:59 -0000 1.2
+++ tokenizer.c 16 Apr 2005 01:45:05 -0000
@@ -1139,6 +1139,9 @@
}
while (c != EOF && c != '\n')
c = tok_nextc(tok);
+ if (c == EOF) {
+ c = '\n';
+ }
}
/* Check for EOF and errors now */
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184112&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
