[issue2216] Problems using logging module with idle
New submission from Andrea Griffini:
I'm not a user of idle, but when asked about a strange behaviour of the
logging module I digged a bit and found what I think is indeed a problem
in the module itself.
The problem is visible if the module is used from idle (or any other IDE
that keeps the same python instance running for multiple execution of
user code); if you call basicConfig(filename="foo.txt"), do some logging
and call shutdown() the handler is closed (correctly closing the file)
but remains attached to the root logger, and gets called again at next
run (giving an error for invalid I/O on a closed file).
This can also be reproduced in a single run with
import logging
logging.basicConfig(filename="logtest.txt")
logging.warning("This is a test")
logging.shutdown()
logging.basicConfig(filename="logtest2.txt")
logging.warning("This is a test (2)")
logging.shutdown()
I think that it is not correct to close the handler but leave it in
place, so I tried patching the module so that every handler keeps a list
of all loggers it is attached to, and removes itself from loggers at
close() time. This way the above code runs correctly (creating two
files) and the logging module still passes the test suite.
I must however admit that logging logic is a bit beyond my grasp
(including a close() call commented out, some uses of lock/release I
don't understand) so may be the attached patch is not correct even if
passes the test.
--
components: Library (Lib)
files: logging.patch
keywords: patch
messages: 63179
nosy: ag6502
severity: normal
status: open
title: Problems using logging module with idle
type: behavior
versions: Python 2.4, Python 2.5, Python 2.6
Added file: http://bugs.python.org/file9583/logging.patch
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2216>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2216] Problems using logging module with idle
Andrea Griffini added the comment: I thougt it was a bug because when calling close() handlers are removed from some data structure (the global dictionary and the global list) but they're left inside the loggers they're attached to. Now I understand that this is a responsibility of who attached the handler; probably my patch would break or just behave differently with code that already managed to remove logging handlers from loggers explicitly or that relied on the fact that even a "closed" handler can still be notified. Having the logging module to work correctly in IDLE and other environments that keep a running instance of the interpreter provided that shutdown is called would have been just a lucky nice side effect of "fixing" handler.close (of course those IDEs will still have potential problems with any module that keeps an internal state). __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2216> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18156] Add an 'attr' attribute to AttributeError
Andrea Griffini added the comment: Even porting to the new wonderful 'attr' field is not going to make the code correct... (the exception could be bubbling up from code down ten frames about a different unrelated attribute that happens to have the same name in a different object). BTW cpython has a lot of those "except AttributeError" fragments coded in C with PyErr_ExceptionMatches. -- ___ Python tracker <http://bugs.python.org/issue18156> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16547] IDLE raises an exception in tkinter after fresh file's text has been rendered
Andrea Griffini added the comment: The error cannot be reproduced on 2.7, 3.3 or 3.4 because the problem has been fixed with 1e5e497ee33b (issue 17614) -- nosy: +ag6502 ___ Python tracker <http://bugs.python.org/issue16547> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15274] Patch for issue 5765: stack overflow evaluating eval("()" * 30000)
New submission from Andrea Griffini :
This is a fix for issue #5765: stack overflow evaluating eval("()" * 3)
The solution was to add two fields (recursion_depth and
recursion_limit) to the symbol table object and just increment and
check the depth in symtable_visit_expr raising a RuntimeError in case
the limit is exceeded.
The test case added also covers other similar cases (a.b.b.b.b.b...
and a[0][0][0][0])
There is no depth check in when visiting statement because I cannot
think to a way to nesting statements too much without getting other
errors before. If there is a way to do it, it would be trivial to also
cover that part.
The patch uses the current depth and current recursion limit but
multiplies them for a factor because I suppose that the amount of C
stack used by the compiler per recursion is smaller than the amount
used by the interpreter; the constant in the patch is 4. Using a
constant of 1 (i.e. just using the normal recursion limit) doesn't
allow a previously existing test about big expressions to pass.
--
files: compiler_recursion_limit_check.patch
keywords: patch
messages: 164839
nosy: ag6502
priority: normal
severity: normal
status: open
title: Patch for issue 5765: stack overflow evaluating eval("()" * 3)
Added file:
http://bugs.python.org/file26292/compiler_recursion_limit_check.patch
___
Python tracker
<http://bugs.python.org/issue15274>
___diff -r d9c98730e2e8 Include/symtable.h
--- a/Include/symtable.hSat Jul 07 13:34:50 2012 +1000
+++ b/Include/symtable.hSat Jul 07 14:39:38 2012 +0200
@@ -30,6 +30,8 @@
PyObject *st_private; /* name of current class or NULL */
PyFutureFeatures *st_future;/* module's future features that affect
the symbol table */
+int recursion_depth;/* current recursion depth */
+int recursion_limit;/* recursion limit */
};
typedef struct _symtable_entry {
diff -r d9c98730e2e8 Lib/test/test_compile.py
--- a/Lib/test/test_compile.py Sat Jul 07 13:34:50 2012 +1000
+++ b/Lib/test/test_compile.py Sat Jul 07 14:39:38 2012 +0200
@@ -474,6 +474,14 @@
self.assertInvalidSingle('f()\nxy # blah\nblah()')
self.assertInvalidSingle('x = 5 # comment\nx = 6\n')
+def test_compiler_recursion_limit(self):
+self.assertRaises(RuntimeError, self.compile_single, "()" * 10)
+self.assertRaises(RuntimeError, self.compile_single, "a" + ".b" *
10)
+self.assertRaises(RuntimeError, self.compile_single, "a" + "[0]" *
10)
+self.compile_single("()" * 2000)
+self.compile_single("a" + ".b" * 2000)
+self.compile_single("a" + "[0]" * 2000)
+
def test_main():
support.run_unittest(TestSpecifics)
diff -r d9c98730e2e8 Python/symtable.c
--- a/Python/symtable.c Sat Jul 07 13:34:50 2012 +1000
+++ b/Python/symtable.c Sat Jul 07 14:39:38 2012 +0200
@@ -218,17 +218,37 @@
return NULL;
}
+/* When compiling the use of C stack is probably going to be a lot
+ lighter than when executing Python code but still can overflow
+ and causing a Python crash if not checked (e.g. eval("()"*30)).
+ Using the current recursion limit for the compiler too seems
+ overconstraining so a factor is used to allow deeper recursion
+ when compiling an expression.
+*/
+#define COMPILER_STACK_FRAME_SCALE 4
+
struct symtable *
PySymtable_Build(mod_ty mod, const char *filename, PyFutureFeatures *future)
{
struct symtable *st = symtable_new();
asdl_seq *seq;
int i;
+PyThreadState *tstate;
if (st == NULL)
return st;
st->st_filename = filename;
st->st_future = future;
+
+/* Setup recursion depth check counters */
+tstate = PyThreadState_GET();
+if (!tstate) {
+PySymtable_Free(st);
+return NULL;
+}
+st->recursion_depth = tstate->recursion_depth * COMPILER_STACK_FRAME_SCALE;
+st->recursion_limit = Py_GetRecursionLimit() * COMPILER_STACK_FRAME_SCALE;
+
/* Make the initial symbol information gathering pass */
if (!GET_IDENTIFIER(top) ||
!symtable_enter_block(st, top, ModuleBlock, (void *)mod, 0, 0)) {
@@ -1268,6 +1288,12 @@
static int
symtable_visit_expr(struct symtable *st, expr_ty e)
{
+if (++st->recursion_depth > st->recursion_limit) {
+PyErr_SetString(PyExc_RuntimeError,
+"maximum recursion depth exceeded while compiling an
expression");
+--st->recursion_depth;
+return 0;
+}
switch (e->kind) {
case BoolOp_kind:
VISIT_SEQ(st, expr, e->v.BoolOp.values);
@@ -1280,8 +1306,10 @@
VISIT(st, expr, e->v.UnaryOp.o
[issue5765] stack overflow evaluating eval("()" * 30000)
Andrea Griffini added the comment: This is a fix for this issue. The solution was to add two fields (recursion_depth and recursion_limit) to the symbol table object and just increment and check the depth in symtable_visit_expr raising a RuntimeError in case the limit is exceeded. The test case added also covers other similar cases (a.b.b.b.b.b... and a[0][0][0][0]) There is no depth check in when visiting statement because I cannot think to a way to nesting statements too much without getting other errors before. If there is a way to do it, it would be trivial to also cover that part. The patch uses the current depth and current recursion limit but multiplies them for a factor because I suppose that the amount of C stack used by the compiler per recursion is smaller than the amount used by the interpreter; the constant in the patch is 4. Using a constant of 1 (i.e. just using the normal recursion limit) doesn't allow a previously existing test about big expressions to pass. -- keywords: +patch nosy: +ag6502 Added file: http://bugs.python.org/file26293/compiler_recursion_limit_check.patch ___ Python tracker <http://bugs.python.org/issue5765> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15274] Patch for issue 5765: stack overflow evaluating eval("()" * 30000)
Andrea Griffini added the comment: I sent an email because I was not able to log in. The patch has been submitted to the correct issue (6765). -- resolution: -> duplicate status: open -> closed ___ Python tracker <http://bugs.python.org/issue15274> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1616125] Cached globals+builtins lookup optimization
Andrea Griffini added the comment: Closing as it was a partial implementation of a bad idea with questionable gains. -- resolution: -> invalid status: open -> closed ___ Python tracker <http://bugs.python.org/issue1616125> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5765] stack overflow evaluating eval("()" * 30000)
Andrea Griffini added the comment:
I missed all the macrology present :-( ... the following is a patch that takes
it into account (also defines a VISIT_QUIT macro to make more visible the exit
points). The handling has been also extended to visit_stmt because the macros
are shared.
Of course all this makes sense assuming that a cleanup in case of error is
indeed desired...
BTW: shouldn't be all those statement macros of the "do{...}while(0)" form
instead of just being wrapped in "{}" ? I see potential problems with if/else...
--
Added file:
http://bugs.python.org/file26913/compiler_recursion_limit_check_2.patch
___
Python tracker
<http://bugs.python.org/issue5765>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5765] stack overflow evaluating eval("()" * 30000)
Andrea Griffini added the comment: On Mon, Aug 20, 2012 at 12:27 AM, Antoine Pitrou wrote: > Indeed I don't like the introduction of COMPILER_STACK_FRAME_SCALE. > Re-using the existing infrastructure would be much easier to maintain. > The default recursion limit is 1000, which should cover any non-pathological > code, IMHO. I submitted a new version with the scale lowered to 3. Using a lower value (e.g. 1) however makes "test_extended_args" fail (the test tries to compile an expression with 2500+ terms). If it's ok to make that test to throw instead then the whole scaling could be removed. -- ___ Python tracker <http://bugs.python.org/issue5765> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
