[ python-Bugs-1333982 ] Bugs of the new AST compiler
Bugs item #1333982, was opened at 2005-10-21 03:08 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1333982&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Armin Rigo (arigo) Assigned to: Nobody/Anonymous (nobody) Summary: Bugs of the new AST compiler Initial Comment: The newly merged AST branch is likely to expose a number of small problems before it stabilizes, so here is a tentative bug tracker entry to collect such small problems. -- >Comment By: Neal Norwitz (nnorwitz) Date: 2005-10-23 12:14 Message: Logged In: YES user_id=33168 Checkpoint of outstanding issues. I think all the others mentioned so far have been fixed: * Raymond's warnings in compile.c (unused locals are fixed) * EXTENDED_ARG problem * LOAD_CONST/POP_TOP (note we can fix this in the optimizer generally which would get rid of other useless code too) -- Comment By: Raymond Hettinger (rhettinger) Date: 2005-10-22 21:53 Message: Logged In: YES user_id=80475 FWIW, here are the warnings issued by my compiler: Python-ast.c C:\py25\Python\Python-ast.c(1995) : warning C4101: 'i' : unreferenced local variable C:\py25\Python\Python-ast.c(2070) : warning C4101: 'i' : unreferenced local variable C:\py25\Python\Python-ast.c(2085) : warning C4101: 'i' : unreferenced local variable C:\py25\Python\Python-ast.c(2130) : warning C4101: 'i' : unreferenced local variable C:\py25\Python\Python-ast.c(2151) : warning C4101: 'i' : unreferenced local variable C:\py25\Python\Python-ast.c(2261) : warning C4101: 'i' : unreferenced local variable C:\py25\Python\Python-ast.c(2270) : warning C4101: 'i' : unreferenced local variable compile.c C:\py25\Python\compile.c(3782) : warning C4305: '=' : truncation from 'const int ' to 'char ' C:\py25\Python\compile.c(3802) : warning C4305: '=' : truncation from 'const int ' to 'char ' C:\py25\Python\compile.c(3806) : warning C4305: '=' : truncation from 'const int ' to 'char ' -- Comment By: Neal Norwitz (nnorwitz) Date: 2005-10-22 00:28 Message: Logged In: YES user_id=33168 I assigned to Jeremy and Neil in the hopes they will see this message and know about these problems. -- Comment By: Armin Rigo (arigo) Date: 2005-10-21 05:45 Message: Logged In: YES user_id=4771 The following (similarly strange-looking) code snippets compiled successfully before, now they give SyntaxErrors: def f(): class g: exec "hi" x def f(x): class g: exec "hi" x def f(): class g: from a import * x def f(x): class g: from a import * x -- Comment By: Armin Rigo (arigo) Date: 2005-10-21 05:33 Message: Logged In: YES user_id=4771 I would suspect the following one to be due to incorrect handling of EXTENDED_ARG -- it's from a PyPy test about that: longexpr = 'x = x or ' + '-x' * 2500 code = ''' def f(x): %s %s %s %s %s %s %s %s %s %s while x: x -= 1 # EXTENDED_ARG/JUMP_ABSOLUTE here return x ''' % ((longexpr,)*10) exec code f(5) SystemError: unknown opcode dis.dis() shows that the target of both the SETUP_LOOP and the JUMP_IF_FALSE at the start of the loop are wrong. -- Comment By: Armin Rigo (arigo) Date: 2005-10-21 05:15 Message: Logged In: YES user_id=4771 The Python rules about which names get mangled are a bit insane. I share mwh's view that mangling should never have been invented in the first place, but well: >>> def f(): ... __x = 5 ... class X: ... def g(self): ... return __x ... return X ... Fatal Python error: unknown scope for _X__x in X(135832776) in -- Comment By: Armin Rigo (arigo) Date: 2005-10-21 05:01 Message: Logged In: YES user_id=4771 For reference, an optimization that got lost: def f(): 'a' 'b' 'a' is the docstring, but the 'b' previously did not show up anywhere in the code object. Now there is the LOAD_CONST/POP_TOP pair. -- Comment By: Armin Rigo (arigo) Date: 2005-10-21 04:54 Message: Logged In: YES user_id=4771 any reason why lambda function
[ python-Bugs-1289136 ] distutils extension library path bug on cygwin
Bugs item #1289136, was opened at 2005-09-12 15:04
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1289136&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: John Whitley (jwhitley)
Assigned to: Nobody/Anonymous (nobody)
Summary: distutils extension library path bug on cygwin
Initial Comment:
A while back I reported a problem on the Cygwin mailing
list where all python extension packages would fail
"python setup.py install" at the link step due to a
mangled lib dir (-L) option. distutils was producing a
link line with "-L.", instead of the desired
"-L/usr/lib/python2.4/config". I've finally rooted out
the cause of this problem.
The relevant code is the if-block starting at line 188 of:
/usr/lib/python2.4/distutils/command/build_ext.py
I've reproduced that block here for clarity of
discussion (indentation truncated for redability)
if sys.platform[:6] == 'cygwin' or sys.platform[:6]
== 'atheos':
if string.find(sys.executable, sys.exec_prefix)
!= -1:
# building third party extensions
self.library_dirs.append(os.path.join(sys.prefix, "lib",
"python" +
get_python_version(),
"config"))
else:
# building python standard extensions
self.library_dirs.append('.')
The test "string.find(...) != -1" attempts to test
whether "/usr" appears in the full executable name.
This incorrectly fails in the case that /bin is in the
user's path before /usr/bin. (i.e.
string.find("/bin/python","/usr") == -1) Note that a
vagary of Cygwin is that /usr/bin is a Cygwin mount to
/bin.
The user-side workaround is to ensure that /usr/bin
appears in your path before /bin. It looks like a new
and improved Cygwin special case test is needed to fix
this problem; I'm not sure offhand what the best case
would be. Perhaps an outright test as follows would work:
sys.executable.startswith(sys.exec_prefix) or
sys.executable.startswith(os.sep+"bin")
--
>Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-23 19:47
Message:
Logged In: YES
user_id=33168
What does os.readlink(sys.executable) return?
Maybe that will help you find the true canonical name?
--
Comment By: Jason Tishler (jlt63)
Date: 2005-09-16 05:04
Message:
Logged In: YES
user_id=86216
John,
Thanks for the excellent analysis!
All,
Unfortunately, I'm not sure what is the best way to solve this
problem. Does anyone have any suggestions? Thanks.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1289136&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1246473 ] line numbers off by 1 in dis
Bugs item #1246473, was opened at 2005-07-27 19:18 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1246473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: line numbers off by 1 in dis Initial Comment: test_dis.py:test_dis is failing for two reasons: the line numbers are off by one (need to be bumped up) and the peepholer is not being used and thus the LOAD_CONST(None);RETURN_VALUE idiom is still being tacked on to everything. The former problem needs to be fixed while the latter is not an issue and thus the test needs to get tweaked to allow for the return. -- >Comment By: Neal Norwitz (nnorwitz) Date: 2005-10-23 19:54 Message: Logged In: YES user_id=33168 Has this report been overcome by events? Is it still relevant? If so, what about the bug report mentioned below? -- Comment By: Brett Cannon (bcannon) Date: 2005-10-07 13:05 Message: Logged In: YES user_id=357491 Maybe, but what would that buy us? We can make the attribute optional on expression nodes and slowly work it all in, but it will need to be done at some point. We could make this the last major fix and deal with everything else that does not rely on this since it is a rather shallow fix in and of itself. -- Comment By: Neil Schemenauer (nascheme) Date: 2005-10-07 12:36 Message: Logged In: YES user_id=35752 As you say, this looks like it will be tedious to fix. Maybe we could change the ast node constructors to take lineno as an optional argument (e.g. using va_list). -- Comment By: Brett Cannon (bcannon) Date: 2005-07-27 22:51 Message: Logged In: YES user_id=357491 Rev. 1.1.2.110 of Python/newcompile.c has the fix for test_dis:test_dis. The more thinking I do about it the more I realize that expression nodes in the AST will need to keep a lineno attribute in order to have lnotab to be set properly. That will not be a fun patch to do. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-27 22:25 Message: Logged In: YES user_id=357491 I don't think this last problem is easily fixed. If you look at the AST, only statements have a lineno attribute. But it is an expression that is on another line from the main line of the statement. I cannot think of any way to get the line information from the statement without adding a lineno attribute to expressions and removing it from statements. That should give the coverage needed to know when another line is being outputted. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-27 20:59 Message: Logged In: YES user_id=357491 I have test_dis now passing thanks to some refactoring to handle the initial line. Now I just need to try to fix test_bug_708901. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-27 19:40 Message: Logged In: YES user_id=357491 OK, so first oddity; in both Python 2.5 and the AST branch co_firstlineno, when working on a function at the interpreter prompt, is set to 1. But if you look at co_lnotab, AST has '\x05\x01' while 2.5 has '\x00\x01\x05\x01'. If you tack on another line to the test function, AST has '\x05\x01\x05\x01' while 2.5 has '\x00\x01\x05\x01\x05\x01'. The critical thing is the initial '\x00\x01'. Looks like 2.5 tacks that on or somehow decides it needs to state the initial line at 0 is a byte long. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1246473&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1246473 ] line numbers off by 1 in dis
Bugs item #1246473, was opened at 2005-07-28 02:18 Message generated for change (Settings changed) made by nascheme You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1246473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: AST >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Brett Cannon (bcannon) >Assigned to: Neil Schemenauer (nascheme) Summary: line numbers off by 1 in dis Initial Comment: test_dis.py:test_dis is failing for two reasons: the line numbers are off by one (need to be bumped up) and the peepholer is not being used and thus the LOAD_CONST(None);RETURN_VALUE idiom is still being tacked on to everything. The former problem needs to be fixed while the latter is not an issue and thus the test needs to get tweaked to allow for the return. -- Comment By: Neal Norwitz (nnorwitz) Date: 2005-10-24 02:54 Message: Logged In: YES user_id=33168 Has this report been overcome by events? Is it still relevant? If so, what about the bug report mentioned below? -- Comment By: Brett Cannon (bcannon) Date: 2005-10-07 20:05 Message: Logged In: YES user_id=357491 Maybe, but what would that buy us? We can make the attribute optional on expression nodes and slowly work it all in, but it will need to be done at some point. We could make this the last major fix and deal with everything else that does not rely on this since it is a rather shallow fix in and of itself. -- Comment By: Neil Schemenauer (nascheme) Date: 2005-10-07 19:36 Message: Logged In: YES user_id=35752 As you say, this looks like it will be tedious to fix. Maybe we could change the ast node constructors to take lineno as an optional argument (e.g. using va_list). -- Comment By: Brett Cannon (bcannon) Date: 2005-07-28 05:51 Message: Logged In: YES user_id=357491 Rev. 1.1.2.110 of Python/newcompile.c has the fix for test_dis:test_dis. The more thinking I do about it the more I realize that expression nodes in the AST will need to keep a lineno attribute in order to have lnotab to be set properly. That will not be a fun patch to do. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-28 05:25 Message: Logged In: YES user_id=357491 I don't think this last problem is easily fixed. If you look at the AST, only statements have a lineno attribute. But it is an expression that is on another line from the main line of the statement. I cannot think of any way to get the line information from the statement without adding a lineno attribute to expressions and removing it from statements. That should give the coverage needed to know when another line is being outputted. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-28 03:59 Message: Logged In: YES user_id=357491 I have test_dis now passing thanks to some refactoring to handle the initial line. Now I just need to try to fix test_bug_708901. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-28 02:40 Message: Logged In: YES user_id=357491 OK, so first oddity; in both Python 2.5 and the AST branch co_firstlineno, when working on a function at the interpreter prompt, is set to 1. But if you look at co_lnotab, AST has '\x05\x01' while 2.5 has '\x00\x01\x05\x01'. If you tack on another line to the test function, AST has '\x05\x01\x05\x01' while 2.5 has '\x00\x01\x05\x01\x05\x01'. The critical thing is the initial '\x00\x01'. Looks like 2.5 tacks that on or somehow decides it needs to state the initial line at 0 is a byte long. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1246473&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1191458 ] [AST] Failing tests
Bugs item #1191458, was opened at 2005-04-28 03:30 Message generated for change (Comment added) made by nascheme You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191458&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: AST Status: Open Resolution: None >Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Jeremy Hylton (jhylton) Summary: [AST] Failing tests Initial Comment: This tracker item is to be used to keep track of what tests are currently failing on the AST branch. This is somewhat important sinced there are so many failures it can be hard to detect if a change fixed what it was supposed to or actually managed to break more code. When posting follow-ups of fixed tests, please re-list the currently failing tests so as to make it simple to reference the current state of things. So, the current offenders are (from a straight ``regrtest.py`` run on my OS X box, which always has test__locale fail so I removed it):: test_compile test_cookielib test_dis test_doctest test_future test_genexps test_inspect test_itertools test_new test_peepholer test_scope test_socket test_sort test_subprocess test_symtable test_syntax test_trace test_traceback test_warnings test_zipfile -- >Comment By: Neil Schemenauer (nascheme) Date: 2005-10-24 04:51 Message: Logged In: YES user_id=35752 I believe the only remaining broken tests are the ones commented out in test_trace. -- Comment By: Jeremy Hylton (jhylton) Date: 2005-10-07 18:46 Message: Logged In: YES user_id=31392 test_dis update: Fixed one of two failing tests. test_dis() was easy. test_bug_708901() is harder. The current AST only stores line numbers for statements. The test case is checking that the lnotab can assign a single statement multiple line numbers; in this case, the statement spans multiple lines and the disassembled code should reflect that. The fix is probably to add line numbers to expressions, which requires changing the AST definition and updating ast.c to assign line numbers. -- Comment By: Jeremy Hylton (jhylton) Date: 2005-10-07 18:01 Message: Logged In: YES user_id=31392 Here's a quick status report on Linux + GCC 3.2.2 from today: 12 tests failed: test_dis test_doctest test_future test_genexps test_inspect test_new test_peepholer test_pwd test_scope test_subprocess test_symtable test_trace 7 skips unexpected on linux2: test_hotshot test_bsddb test_email test_parser test_transformer test_email_codecs test_compiler I'm going to trace into the details of why each of these tests is failing. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-11 04:15 Message: Logged In: YES user_id=357491 Scratch the "all open bugs" comment; didn't get to bug #1195576. -- Comment By: Brett Cannon (bcannon) Date: 2005-07-11 04:13 Message: Logged In: YES user_id=357491 After applying all patches associated with all open bugs, the failing tests from ``./python.exe Lib/test/regrtest.py`` are: 14 tests failed: test_cookielib test_dis test_future test_genexps test_inspect test_new test_peepholer test_scope test_subprocess test_symtable test_syntax test_trace test_traceback test_warnings -- Comment By: Nick Coghlan (ncoghlan) Date: 2005-05-28 11:40 Message: Logged In: YES user_id=1038590 Fixing #1186353 (lambda argument unpacking) fixes test_sort and test_itertools: 17 tests failed: test_compile test_cookielib test_dis test_doctest test_future test_genexps test_inspect test_new test_ossaudiodev test_peepholer test_scope test_subprocess test_symtable test_syntax test_trace test_traceback test_warnings -- Comment By: Nick Coghlan (ncoghlan) Date: 2005-05-28 08:33 Message: Logged In: YES user_id=1038590 Bumped bug priority to 9 so it stays on top of the SF tracker. Also, we can't merge the AST branch until this bug is closed, so it is fairly important to close out these issues :) Running "./python Lib/test/regrtest.py -uall" Suse Linux 9.1: 20 tests failed: test_compile test_cookielib test_dis test_doctest test_future test_genexps test_inspect test_itertools test_new test_ossaudiodev test_peepholer test_scope test_sort test_subprocess test_symtable test_syntax test_trace test_traceback test_warnings test_zipfile WIth the patch for 1186195
