[ python-Bugs-1445210 ] embedding Python causes memory leaks
Bugs item #1445210, was opened at 2006-03-08 00:20
Message generated for change (Comment added) made by loewis
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1445210&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Andrew Trevorrow (andykt)
Assigned to: Nobody/Anonymous (nobody)
Summary: embedding Python causes memory leaks
Initial Comment:
[This bug has been submitted by others but for some reason it
has been marked Closed. I consider it to be an extremely serious
bug -- if I can't solve it I'm going to have to abandon Python as
my app's scripting language, even though I've fallen in love!]
I've added Python script support to my cross-platfom wxWidgets app
so that users can run .py scripts from within the app to automate the
GUI and do other fancy things. It all works very nicely, except for
one nasty problem: *every* time a script is run there is a memory leak,
usually small (about 10K) but sometimes massive (about 4MB in the
case of one rather complicated script).
The problem occurs on both Mac OS 10.3.9 and Windows 2000.
I'm using Python 2.3 on the Mac and 2.4.2 on Windows.
Every time the user runs a script, my app makes these calls:
(I've removed a lot of irrelevant stuff.)
Py_Initialize();
PyRun_SimpleString("execfile('foo.py')");
Py_Finalize();
It's definitely not a wxWidgets problem. In fact it's quite easy to
see the memory leak using a simple command-line program:
#include
#include
main(int argc, char *argv[])
{
int i;
for (i=0; i<1000; i++) {
Py_Initialize();
Py_Finalize();
printf(".");
if ((i+1) % 50 == 0) printf("\n");
}
}
Note that it doesn't even execute a script. If I run this program on
my Mac and watch its memory usage with Activity Monitor, I see a leak
of about 10K each time through the loop. Similar result on Windows.
Curiously, on both machines, the Py_Finalize() call takes longer and
longer to complete whatever it's doing. The above program takes a
few *minutes* to complete on my 400MHz Mac.
Andrew
--
>Comment By: Martin v. Löwis (loewis)
Date: 2006-04-13 09:29
Message:
Logged In: YES
user_id=21627
That the documentation claims Py_Finalize releases all
memory is a bug; I just fixed this in r45344. The original
problem cannot be fixed (atleast not until Python 3000);
closing it as "won't fix".
--
Comment By: Andrew Trevorrow (andykt)
Date: 2006-03-10 06:43
Message:
Logged In: YES
user_id=1281947
See http://evanjones.ca/python-memory.html for some
useful info. Apparently the Python memory allocator never
releases memory back to the OS! So if a complicated script
happens to consume 100MB of interpreter memory then
that amount is no longer available to the app in which
Python is embedded. Even worse, if a script has a
(Python) memory leak then there's nothing the app can
do about it. It would be great if wrapping each script
inside Py_Initialize/Py_Finalize could avoid all that.
There should be some way to tell Python "release all
the memory you've ever allocated and start again
with a clean slate".
--
Comment By: Andrew Trevorrow (andykt)
Date: 2006-03-08 10:50
Message:
Logged In: YES
user_id=1281947
Bloody hell -- sorry for the bad line breaks.
One day I'll figure out how to use sf properly!
--
Comment By: Andrew Trevorrow (andykt)
Date: 2006-03-08 10:46
Message:
Logged In: YES
user_id=1281947
> Why do you call Py_Initialize/Py_Finalize more than once?
How else do I tell Python to free up memory after each PyRun_SimpleString
call?
I want users to be able to run scripts many times from within my app.
If I just keep calling PyRun_SimpleString then my app will leak more and
more memory until it becomes unusable.
>From http://docs.python.org/api/embedding.html:
Sometimes, it is desirable to ``uninitialize'' Python. For instance, the
application may want to start over (make another call to Py_Initialize())
or the application is simply done with its use of Python and wants to free
all memory allocated by Python. This can be accomplished by calling
Py_Finalize().
That's exactly what I want to do. I want the interpreter to run a script
and then release all the resources used by that script. Unfortunately,
Py_Finalize does *not* restore memory usage to what it was before the
Py_Initialize call. I wouldn't mind if there was a one-off allocation
cost (the 1st time Py_Initialize is called), but my app is leaking more
memory *every* time a script
[ python-Bugs-1469629 ] __dict__ = self in subclass of dict causes a memory leak?
Bugs item #1469629, was opened at 2006-04-13 05:04 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1469629&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Dobes V (dobesv) Assigned to: Nobody/Anonymous (nobody) Summary: __dict__ = self in subclass of dict causes a memory leak? Initial Comment: Using: ActivePython 2.4.2 Build 10 (ActiveState Corp.) based on Python 2.4.2 (#67, Jan 17 2006, 15:36:03) [MSC v.1310 32 bit (Intel)] on win32 For reasons I do not understand, the following class leaks itself continuously: class AttrDict(dict): def __init__(self, *args, **kw): dict.__init__(self, *args, **kw) self.__dict__ = self Whereas this version does not: class AttrDict(dict): def __init__(self, *args, **kw): dict.__init__(self, *args, **kw) def __getattr__(self, key): return self[key] def __setattr__(self, key, value): self[key] = value My test looks like this: for n in xrange(100): import gc gc.collect() ad = AttrDict() ad['x'] = n ad.y = ad.x print n, ad.x, ad.y And I sit and watch in the windows task manager while the process grows and grows. With the __getattr__ version, it doesn't grow. -- >Comment By: Armin Rigo (arigo) Date: 2006-04-13 09:58 Message: Logged In: YES user_id=4771 This is caused by the tp_clear not doing its job -- in this case, tp_clear is subtype_clear(), which does not reset the __dict__ slot to NULL because it assumes that the __dict__ slot's content itself will be cleared, which is perfectly true but doesn't help if self.__dict__ is self. Attached a patch to fix this. It's kind of hard to test for this bug because all instances of AttrDict are really cleared, weakrefs to them are removed, etc. Also attached is an example showing a similar bug: a cycle through the ob_type field, with a object U whose class is itself. It is harder to clear this link because we cannot just set ob_type to NULL in subtype_clear. Ideas welcome... -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1469629&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1464056 ] curses library in python 2.4.3 broken
Bugs item #1464056, was opened at 2006-04-04 14:17
Message generated for change (Comment added) made by vnainar
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464056&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: nomind (vnainar)
Assigned to: Nobody/Anonymous (nobody)
Summary: curses library in python 2.4.3 broken
Initial Comment:
My python programs using curses library do not work
with version 2.4.3.Even the 'ncurses.py ' demo program
in the Demo/curses directory does not work correctly.
The problem seems to be in the 'panels' library
--
>Comment By: nomind (vnainar)
Date: 2006-04-13 16:30
Message:
Logged In: YES
user_id=1493752
Well , it is the linux console (in VGA mode ).
The local is en_US.The size of the console is
100X37.
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-04-11 11:02
Message:
Logged In: YES
user_id=21627
Ah, ok. vnainar, atler_: What terminal had you been using to
make it crash? What is your locale? Any other conditions
that might be relevant (e.g. dimension of the terminal)?
--
Comment By: nomind (vnainar)
Date: 2006-04-11 10:43
Message:
Logged In: YES
user_id=1493752
Removing 'ncursesw' (there are two references to it in
'setup.py') seems to solve the problem. I noticed one more
oddity. Even before the above recompilation , it ran fine
on an Xterm !
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-04-11 03:58
Message:
Logged In: YES
user_id=21627
That's hard to tell. Somebody would have to debug ncurses to
find out why it crashes. Notice that it crashes only on some
installations, so it is likely rather a problem with your
ncurses installation, than with Python (or even with ncurses
itself).
--
Comment By: Jan Palus (atler_)
Date: 2006-04-11 02:54
Message:
Logged In: YES
user_id=1497957
loewis: removing lines refering to ncursesw solves the
problem. ncurses.py runs fine as well as other programs.
What is actual problem then? Something with ncursesw or
with python?
Anyway, Thanks for your help.
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-04-10 02:28
Message:
Logged In: YES
user_id=21627
atler_: around line 427, you find
if self.compiler.find_library_file(lib_dirs,
'ncursesw'):
readline_libs.append('ncursesw')
elif self.compiler.find_library_file(lib_dirs,
'ncurses'):
Replace that with
if self.compiler.find_library_file(lib_dirs,
'ncurses'):
(i.e. dropping the ncursesw part), and rebuild.
--
Comment By: Jan Palus (atler_)
Date: 2006-04-09 19:18
Message:
Logged In: YES
user_id=1497957
More complete backtrace, I hope it will help:
http://pastebin.com/649445
--
Comment By: Anthony Baxter (anthonybaxter)
Date: 2006-04-09 17:44
Message:
Logged In: YES
user_id=29957
The buildbot boxes don't show this problem.
You might need to rebuild a Python with -g and unstripped to
get a useful backtrace.
--
Comment By: Jan Palus (atler_)
Date: 2006-04-09 16:21
Message:
Logged In: YES
user_id=1497957
$ ldd /usr/lib/python2.4/lib-dynload/_curses.so
libncursesw.so.5 => /usr/lib/libncursesw.so.5
(0x7004c000)
libpthread.so.0 => /lib/libpthread.so.0 (0x7008)
libc.so.6 => /lib/libc.so.6 (0x700e4000)
libdl.so.2 => /lib/libdl.so.2 (0x70228000)
libtinfow.so.5 => /usr/lib/libtinfow.so.5 (0x7023c000)
/lib/ld-linux.so.2 (0x7000)
...
Program received signal SIGSEGV, Segmentation fault.
0x7063947c in hide_panel () from /usr/lib/libpanel.so.5
gdb) bt
#0 0x7063947c in hide_panel () from /usr/lib/libpanel.so.5
#1 0x706254b8 in ?? () from
/usr/lib/python2.4/lib-dynload/_curses_panel.so
#2 0x706254b8 in ?? () from
/usr/lib/python2.4/lib-dynload/_curses_panel.so
Previous frame identical to this frame (corrupt stack?)
It seems that only programs using panel library cause
segfaults (all other demos run fine except ncurses.py),
sorry for a mistake in a last post.
loewis: I'm not sure I understand second point. What excatly
s
[ python-Bugs-1464571 ] Changes to generator object's gi_frame attr
Bugs item #1464571, was opened at 2006-04-04 19:44 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Collin Winter (collinwinter) Assigned to: A.M. Kuchling (akuchling) Summary: Changes to generator object's gi_frame attr Initial Comment: In 2.4 (as late as 2.4.3), a generator's gi_frame attribute was always a frame object. In the current SVN revision (43631), gi_frame is sometimes None, a change I can't find any documentation of in the What's New for 2.5 section. If this was intentional, it should be documented -- I can't be the only one with packages that used this behaviour. If it was unintentional, I'd appreciate it if this could be fixed before 2.5 goes out. I don't have a (simple) repro case at this time, but I'll post one as soon as I can simplify the current one down. -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 08:50 Message: Logged In: YES user_id=11375 I've added a mention of this page to the "What's New"; thanks for the suggestion! -- Comment By: iga Seilnacht (zseil) Date: 2006-04-12 19:35 Message: Logged In: YES user_id=1326842 This was changed again in revision 45316. The current comment in genobject.h says: Note: gi_frame can be NULL if the generator is "finished" -- Comment By: Collin Winter (collinwinter) Date: 2006-04-04 19:53 Message: Logged In: YES user_id=1344176 *stops, thinks, comes up with repro case in 30 seconds* This does indeed seem to be intentional (introduced in r39239 during the implementation of PEP 342). To trigger this, all you have to do is run a generator til exhaustion (ie, StopIteration); at this point, the frame is decref'd and gi_frame is set to None. I'd appreciate it if this could be added to the "Porting to 2.5" section of What's New. Sorry for the confusion. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1447031 ] whatsnew 2.5: pep 353, python -m
Bugs item #1447031, was opened at 2006-03-10 02:44 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1447031&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Gene Tani (gtani) Assigned to: A.M. Kuchling (akuchling) Summary: whatsnew 2.5: pep 353, python -m Initial Comment: http://docs.python.org/dev/whatsnew/whatsnew25.html I don't see mention of PEP 353 (indexing with ssize_t) or the python -m switch, as per: http://agiletesting.blogspot.com/2006/02/pycon-notes-part-2-guidos-keynote.html -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 09:01 Message: Logged In: YES user_id=11375 The "What's New" was greatly expanded in March, and there are now sections on these two topics. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1447031&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1464571 ] Changes to generator object's gi_frame attr
Bugs item #1464571, was opened at 2006-04-05 01:44 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Collin Winter (collinwinter) Assigned to: A.M. Kuchling (akuchling) Summary: Changes to generator object's gi_frame attr Initial Comment: In 2.4 (as late as 2.4.3), a generator's gi_frame attribute was always a frame object. In the current SVN revision (43631), gi_frame is sometimes None, a change I can't find any documentation of in the What's New for 2.5 section. If this was intentional, it should be documented -- I can't be the only one with packages that used this behaviour. If it was unintentional, I'd appreciate it if this could be fixed before 2.5 goes out. I don't have a (simple) repro case at this time, but I'll post one as soon as I can simplify the current one down. -- Comment By: iga Seilnacht (zseil) Date: 2006-04-13 15:30 Message: Logged In: YES user_id=1326842 I don't think that your change is correct anymore; Philip J. Eby has changed the behaviour again in revision 45316. The log for his checkin says: Don't set gi_frame to Py_None, use NULL instead, eliminating some insane pointer dereferences. See: http://mail.python.org/pipermail/python-dev/2006-April/063647.html http://mail.python.org/pipermail/python-checkins/2006-April/051180.html for more details. -- Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 14:50 Message: Logged In: YES user_id=11375 I've added a mention of this page to the "What's New"; thanks for the suggestion! -- Comment By: iga Seilnacht (zseil) Date: 2006-04-13 01:35 Message: Logged In: YES user_id=1326842 This was changed again in revision 45316. The current comment in genobject.h says: Note: gi_frame can be NULL if the generator is "finished" -- Comment By: Collin Winter (collinwinter) Date: 2006-04-05 01:53 Message: Logged In: YES user_id=1344176 *stops, thinks, comes up with repro case in 30 seconds* This does indeed seem to be intentional (introduced in r39239 during the implementation of PEP 342). To trigger this, all you have to do is run a generator til exhaustion (ie, StopIteration); at this point, the frame is decref'd and gi_frame is set to None. I'd appreciate it if this could be added to the "Porting to 2.5" section of What's New. Sorry for the confusion. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1464571 ] Changes to generator object's gi_frame attr
Bugs item #1464571, was opened at 2006-04-04 19:44 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Collin Winter (collinwinter) Assigned to: A.M. Kuchling (akuchling) Summary: Changes to generator object's gi_frame attr Initial Comment: In 2.4 (as late as 2.4.3), a generator's gi_frame attribute was always a frame object. In the current SVN revision (43631), gi_frame is sometimes None, a change I can't find any documentation of in the What's New for 2.5 section. If this was intentional, it should be documented -- I can't be the only one with packages that used this behaviour. If it was unintentional, I'd appreciate it if this could be fixed before 2.5 goes out. I don't have a (simple) repro case at this time, but I'll post one as soon as I can simplify the current one down. -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 09:40 Message: Logged In: YES user_id=11375 No, it's correct; I verified it with a small test program. Whether PJE's code uses Py_None or NULL, you still get a None in your Python code. -- Comment By: iga Seilnacht (zseil) Date: 2006-04-13 09:30 Message: Logged In: YES user_id=1326842 I don't think that your change is correct anymore; Philip J. Eby has changed the behaviour again in revision 45316. The log for his checkin says: Don't set gi_frame to Py_None, use NULL instead, eliminating some insane pointer dereferences. See: http://mail.python.org/pipermail/python-dev/2006-April/063647.html http://mail.python.org/pipermail/python-checkins/2006-April/051180.html for more details. -- Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 08:50 Message: Logged In: YES user_id=11375 I've added a mention of this page to the "What's New"; thanks for the suggestion! -- Comment By: iga Seilnacht (zseil) Date: 2006-04-12 19:35 Message: Logged In: YES user_id=1326842 This was changed again in revision 45316. The current comment in genobject.h says: Note: gi_frame can be NULL if the generator is "finished" -- Comment By: Collin Winter (collinwinter) Date: 2006-04-04 19:53 Message: Logged In: YES user_id=1344176 *stops, thinks, comes up with repro case in 30 seconds* This does indeed seem to be intentional (introduced in r39239 during the implementation of PEP 342). To trigger this, all you have to do is run a generator til exhaustion (ie, StopIteration); at this point, the frame is decref'd and gi_frame is set to None. I'd appreciate it if this could be added to the "Porting to 2.5" section of What's New. Sorry for the confusion. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1464571 ] Changes to generator object's gi_frame attr
Bugs item #1464571, was opened at 2006-04-05 01:44 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Collin Winter (collinwinter) Assigned to: A.M. Kuchling (akuchling) Summary: Changes to generator object's gi_frame attr Initial Comment: In 2.4 (as late as 2.4.3), a generator's gi_frame attribute was always a frame object. In the current SVN revision (43631), gi_frame is sometimes None, a change I can't find any documentation of in the What's New for 2.5 section. If this was intentional, it should be documented -- I can't be the only one with packages that used this behaviour. If it was unintentional, I'd appreciate it if this could be fixed before 2.5 goes out. I don't have a (simple) repro case at this time, but I'll post one as soon as I can simplify the current one down. -- Comment By: iga Seilnacht (zseil) Date: 2006-04-13 15:44 Message: Logged In: YES user_id=1326842 Oops, my previous comment is only relevant for C code. Sorry for the noise. -- Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 15:40 Message: Logged In: YES user_id=11375 No, it's correct; I verified it with a small test program. Whether PJE's code uses Py_None or NULL, you still get a None in your Python code. -- Comment By: iga Seilnacht (zseil) Date: 2006-04-13 15:30 Message: Logged In: YES user_id=1326842 I don't think that your change is correct anymore; Philip J. Eby has changed the behaviour again in revision 45316. The log for his checkin says: Don't set gi_frame to Py_None, use NULL instead, eliminating some insane pointer dereferences. See: http://mail.python.org/pipermail/python-dev/2006-April/063647.html http://mail.python.org/pipermail/python-checkins/2006-April/051180.html for more details. -- Comment By: A.M. Kuchling (akuchling) Date: 2006-04-13 14:50 Message: Logged In: YES user_id=11375 I've added a mention of this page to the "What's New"; thanks for the suggestion! -- Comment By: iga Seilnacht (zseil) Date: 2006-04-13 01:35 Message: Logged In: YES user_id=1326842 This was changed again in revision 45316. The current comment in genobject.h says: Note: gi_frame can be NULL if the generator is "finished" -- Comment By: Collin Winter (collinwinter) Date: 2006-04-05 01:53 Message: Logged In: YES user_id=1344176 *stops, thinks, comes up with repro case in 30 seconds* This does indeed seem to be intentional (introduced in r39239 during the implementation of PEP 342). To trigger this, all you have to do is run a generator til exhaustion (ie, StopIteration); at this point, the frame is decref'd and gi_frame is set to None. I'd appreciate it if this could be added to the "Porting to 2.5" section of What's New. Sorry for the confusion. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464571&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1470026 ] distutils.core: link to list of Trove classifiers
Bugs item #1470026, was opened at 2006-04-13 12:13 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1470026&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ansel Halliburton (anseljh) Assigned to: Nobody/Anonymous (nobody) Summary: distutils.core: link to list of Trove classifiers Initial Comment: In 10.1 (distutils.core), it would be nice to have a link to the full list of Trove classifiers. There is already a place for such a link in the table for the setup() method; search for "XXX link to better definition". Here is the link to add: http://www.python.org/pypi?:action=list_classifiers I would say "a list of strings; see http://www.python.org/pypi?:action=list_classifiers for a list of all possible values" -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1470026&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1332852 ] BSD DB test failures for BSD DB 3.2
Bugs item #1332852, was opened at 2005-10-19 21:41
Message generated for change (Comment added) made by greg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1332852&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
>Status: Closed
Resolution: None
Priority: 6
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Gregory P. Smith (greg)
Summary: BSD DB test failures for BSD DB 3.2
Initial Comment:
After a bunch of modifications to build and get bsddb
mostly running, I still get 4 test failures with Python
2.4 and CVS head.
FAIL: test_previous_last_looping
(test.test_bsddb.TestBTree)
--
Traceback (most recent call last):
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 128, in test_previous_last_looping
self.assertSetEquals(items, self.d.items())
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 55, in assertSetEquals
self.assertEqual(Set(seqn1), Set(seqn2))
AssertionError: Set([('e', 'Rossum'), ('r',
'invented'), ('q', 'Guido'), ('w', 'van'), ('t',
'Python'), ('y', None)]) != Set([('e', 'Rossum'), ('r',
'invented'), ('t', 'Python'), ('w', 'van'), ('q',
'Guido'), ('y', '')])
==
FAIL: test_previous_last_looping
(test.test_bsddb.TestHashTable)
--
Traceback (most recent call last):
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 128, in test_previous_last_looping
self.assertSetEquals(items, self.d.items())
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 55, in assertSetEquals
self.assertEqual(Set(seqn1), Set(seqn2))
AssertionError: Set([('e', 'Rossum'), ('r',
'invented'), ('t', 'Python'), ('w', 'van'), ('q',
'Guido'), ('y', None)]) != Set([('e', 'Rossum'), ('r',
'invented'), ('t', 'Python'), ('w', 'van'), ('q',
'Guido'), ('y', '')])
==
FAIL: test_previous_last_looping
(test.test_bsddb.TestBTree_InMemory)
--
Traceback (most recent call last):
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 128, in test_previous_last_looping
self.assertSetEquals(items, self.d.items())
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 55, in assertSetEquals
self.assertEqual(Set(seqn1), Set(seqn2))
AssertionError: Set([('e', 'Rossum'), ('r',
'invented'), ('q', 'Guido'), ('w', 'van'), ('t',
'Python'), ('y', None)]) != Set([('e', 'Rossum'), ('r',
'invented'), ('t', 'Python'), ('w', 'van'), ('q',
'Guido'), ('y', '')])
==
FAIL: test_previous_last_looping
(test.test_bsddb.TestHashTable_InMemory)
--
Traceback (most recent call last):
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 128, in test_previous_last_looping
self.assertSetEquals(items, self.d.items())
File
"/home/neal/build/python/dist/clean/Lib/test/test_bsddb.py",
line 55, in assertSetEquals
self.assertEqual(Set(seqn1), Set(seqn2))
AssertionError: Set([('e', 'Rossum'), ('r',
'invented'), ('t', 'Python'), ('w', 'van'), ('q',
'Guido'), ('y', None)]) != Set([('e', 'Rossum'), ('r',
'invented'), ('t', 'Python'), ('w', 'van'), ('q',
'Guido'), ('y', '')])
--
>Comment By: Gregory P. Smith (greg)
Date: 2006-04-13 12:20
Message:
Logged In: YES
user_id=413
ok i've raised the minimum to 3.3 in python 2.5 trunk. svn
r45368.
--
Comment By: Neal Norwitz (nnorwitz)
Date: 2006-04-12 20:20
Message:
Logged In: YES
user_id=33168
I don't mind and think it's reasonable to set whatever
minimum version for 2.5. I don't think 3.2 support should
be removed from 2.4 though. In 2.4, you can print a message
in setup.py if the version is too old.
I think that's conservative enough and reasonble.
--
Comment By: Gregory P. Smith (greg)
Date: 2006-04-12 12:21
Message:
Logged In: YES
user_id=413
i'd prefer to just not support 3.2 anymore.
mind if i just change setup.py to not accept 3.2?
--
Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-19 22:20
Message:
Logged In: YES
user_id=33168
Oops, that last comment got cut off. I've got db-3.2 and
db-4.1 on my system at least. Not sure why the older one is
picked up rather than the newer one.
---
[ python-Bugs-775414 ] bsddb3 hash craps out with threads
Bugs item #775414, was opened at 2003-07-21 19:29 Message generated for change (Settings changed) made by greg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=775414&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules >Group: Python 2.5 Status: Open Resolution: None >Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Gregory P. Smith (greg) Summary: bsddb3 hash craps out with threads Initial Comment: Richie Hindle presented something like the attached (hammer.py) on the spambayes-dev mailing list. On Win98SE and Win2K w/ Python 2.3c1 I usually see this death pretty quickly: Traceback (most recent call last): File "hammer.py", line 36, in ? main() File "hammer.py", line 33, in main hammer(db) File "hammer.py", line 15, in hammer x = db[str(int(random.random() * 10))] File "C:\CODE\PYTHON\lib\bsddb\__init__.py", line 86, in __getitem__ return self.db[key] bsddb._db.DBRunRecoveryError: (-30982, 'DB_RUNRECOVERY: Fatal error, run database recovery -- fatal region error detected; run recovery') Richie also reported "illegal operation" crashes on Win98SE. It's not clear whether a bsddb3 hash *can* be used with threads like this. If it can't, there's a doc bug. If it should be able to, there's a more serious problem. Note that it looks like hashopen() always merges DB_THREAD into the flags, so the absence of specifying DB_THREAD probably isn't the problem. -- Comment By: Gregory P. Smith (greg) Date: 2005-11-05 08:54 Message: Logged In: YES user_id=413 modifying bsddb/__init__.py to wrap all calls with DeadlockWrap will be a bit of a pita (but would be doable). I've attached an example wrapped_hammer.py that demonstrates the _openDBEnv change as well as DeadlockWrap wrapping to work properly. -- Comment By: Gregory P. Smith (greg) Date: 2005-11-05 08:31 Message: Logged In: YES user_id=413 oh good i see you already suggested the simple thread calling lock_detect that I was about to suggest. :) regardless a thread isn't needed. see dbenv.set_lk_detect which tells BerkeleyDB to run deadlock detection automatically anytime a lock conflict occurs. http://www.sleepycat.com/docs/api_c/env_set_lk_detect.html Just add e.set_lk_detect(db.DB_LOCK_DEFAULT) to bsddb/__init__.py's _openDBEnv() function. That causes hammer.py to get DBLockDeadlockError exceptions as expected (dying if the main thread gets one). No lock_detect thread needed. The bsddb legacy interface in __init__.py could have all of its database accesses wrapped in the bsddb.dbutils.DeadlockWrap function. to prevent this. (testing now) -- Comment By: Mark Hammond (mhammond) Date: 2005-11-03 20:20 Message: Logged In: YES user_id=14198 The db_deadlock program ends up being equivalent to a thread repeatedly calling: dbenv.lock_detect(bsddb.db.DB_LOCK_DEFAULT, 0) For completeness, I attach deadlock_hammer.py - a version that uses yet another thread to perform this lock detection. It also catches the deadlock exceptions, printing but ignoring them. Also, due to the way shutdown is less than graceful, I found I needed to add DB_RECOVER_FATAL to the env flags, otherwise I would often hang on open unless I clobbered the DB directory. On both my box (where it took a little while to see a deadlock) and on a dual-processor box (which provoked it much quicker), this version seems to run forever (although with sporadic performance) -- Comment By: Mark Hammond (mhammond) Date: 2005-11-03 18:00 Message: Logged In: YES user_id=14198 Sadly, I believe bsddb is working "as designed". Quoting from http://www.sleepycat.com/docs/api_c/env_open.html "When the DB_INIT_LOCK flag is specified, it is usually necessary to run a deadlock detector, as well." So I dig into my bsddb build tree, and found db_deadlock.exe. Sure enough, once studly_hammer.py had deadlocked, executing db_deadlock in the DB directory got things running again - although the threads all eventually died with: bsddb._db.DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') Obviously it is PITA to need to run an external daemon, and as Python doesn't distribute db_deadlock.exe, the sleepycat license may mean not all applications are allowed to distribute it. This program also polls for deadlocks, meaning your app may hang as long as the poll period. All in all, it seems to suck :) -- Comment By: Gregory P. Smith (greg) Date: 2003-10-05 18:17 Message: L
[ python-Bugs-1466301 ] ImportError: Module _subprocess not found
Bugs item #1466301, was opened at 2006-04-07 14:41 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1466301&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: 3rd Party >Status: Closed Resolution: Invalid Priority: 5 Submitted By: Africanis (africanis) Assigned to: Nobody/Anonymous (nobody) Summary: ImportError: Module _subprocess not found Initial Comment: I'm using Enthought Python 2.3.5 together with IPython and matplotlib on Windows XP SP2. I had problems importing pylab (in order actually to use matplotlib) until I changed the attached file (see from line 365). I have absolutely no idea what effect this will have on other modules (I'm fairly new to programming), but matplotlib seems to work okay now. Ignorance is bliss... -- Comment By: Martin v. Löwis (loewis) Date: 2006-04-11 13:21 Message: Logged In: YES user_id=21627 Thanks for the patch. I'm not quite sure what you have been doing: Python 2.3.5 did not include the subprocess module at all. So if you got it either from IPython or from Enthought Python, you could report the problem to them. They should either include _subprocess.pyd (as it was released with Python 2.4), or enable the if 0 block, meant for pywin32, or drop subprocess entirely. In any case, this should be a problem for Python 2.4 or (the upcoming) Python 2.5. Closing it as a third-party bug. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1466301&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-764437 ] AF_UNIX sockets do not handle Linux-specific addressing
Bugs item #764437, was opened at 2003-07-02 07:13
Message generated for change (Comment added) made by arigo
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=764437&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Pavel Pergamenshchik (ppergame)
Assigned to: Nobody/Anonymous (nobody)
Summary: AF_UNIX sockets do not handle Linux-specific addressing
Initial Comment:
As described in unix(7) manpage, Linux allows for
special "kernel namespace" AF_UNIX sockets defined.
With such sockets, the first byte of the path is \x00,
and the rest is the address. These sockets do not show
up in the filesystem.
socketmodule.c:makesockaddr (as called by recvfrom)
uses code like
PyString_FromString(a->sun_path)
to retrieve the address. This is incorrect -- on Linux, if
the first byte of a->sun_path is null, the function should
use PyString_FromStringAndSize to retrieve the full 108-
byte buffer.
I am not entirely sure that this is the only thing that
needs to be fixed, but bind() and sendto() work with
these sort of socket paths just fine.
--
>Comment By: Armin Rigo (arigo)
Date: 2006-04-13 20:03
Message:
Logged In: YES
user_id=4771
I'm about to check in a slightly different patch
(also at 1062014) which tries to preserve the length
of the abstract namespace addresses (the kernel saves
this length even though it is not zero-terminated,
so that the names '\x00abc' and '\x00abc\x00' and
'\x00abc\x00\x00' are all different in theory).
The patch no longer exposes UNIX_PATH_MAX because
I'm not sure it's useful any more. If anyone who
is relying on this Linux extension more than myself
has comments, now would be a good time :-)
--
Comment By: Irmen de Jong (irmen)
Date: 2004-11-08 11:03
Message:
Logged In: YES
user_id=129426
Patch is at 1062014
The comments below state that UNIX_PATH_MAX is defined in
sys/un.h, but it isn't. The patch gets it in another
(hopefully portable) way (and also exposes it as a new
constant in the socket module)
--
Comment By: Aaron Brady (insomnike)
Date: 2004-07-07 14:39
Message:
Logged In: YES
user_id=1057404
It should use UNIX_PATH_MAX, but it should also check that
the length of the buffer supplied is correct, my bad.
--
Comment By: Pavel Pergamenshchik (ppergame)
Date: 2004-07-07 14:37
Message:
Logged In: YES
user_id=595126
"""The sockets address in this namespace is given by the
rest of the bytes in sun_path. Note that names in the
abstract namespace are not zero‐terminated."""
The length would be UNIX_PATH_MAX in this case.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2004-07-07 14:04
Message:
Logged In: YES
user_id=11375
Should it use UNIX_PATH_MAX as the length of the string, or
1+ strlen(a->sun_path+1)? (The leading null byte, plus the
following null-terminated string?)
--
Comment By: Aaron Brady (insomnike)
Date: 2004-06-05 19:16
Message:
Logged In: YES
user_id=1057404
Also checks for "linux" to be defined, on Mondragon's
recommendation.
###
--- socketmodule.c 3 Jun 2004 09:24:42 - 1.291
+++ socketmodule.c 5 Jun 2004 18:17:51 -
@@ -942,6 +942,11 @@
case AF_UNIX:
{
struct sockaddr_un *a = (struct sockaddr_un
*) addr;
+#if defined(UNIX_PATH_MAX) && defined(linux)
+ if (*a->sun_path == 0) {
+ return
PyString_FromStringAndSize(a->sun_path, UNIX_PATH_MAX);
+ }
+#endif /* UNIX_PATH_MAX && linux */
return PyString_FromString(a->sun_path);
}
#endif /* AF_UNIX */
###
--
Comment By: Aaron Brady (insomnike)
Date: 2004-06-05 19:06
Message:
Logged In: YES
user_id=1057404
The below patch adds the functionality if UNIX_PATH_MAX is
defined (in Linux it's in sys/un.h).
###
--- socketmodule.c 3 Jun 2004 09:24:42 - 1.291
+++ socketmodule.c 5 Jun 2004 18:08:47 -
@@ -942,6 +942,11 @@
case AF_UNIX:
{
struct sockaddr_un *a = (struct sockaddr_un
*) addr;
+#if defined(UNIX_PATH_MAX)
+ if (*a->sun_path == 0) {
+ return
PyString_FromStringAndSize(a->sun_path, UNIX_PATH_MAX);
+ }
+#endif /* UNIX_PATH_MAX */
return PyString_FromString(a->sun_path);
}
#endif /* AF_UNIX *
[ python-Bugs-1460514 ] ctypes extension does not compile on Mac OS 10.3.9
Bugs item #1460514, was opened at 2006-03-29 01:28 Message generated for change (Comment added) made by dalke You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1460514&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 2.5 Status: Open Resolution: None Priority: 7 Submitted By: Andrew Dalke (dalke) Assigned to: Thomas Heller (theller) Summary: ctypes extension does not compile on Mac OS 10.3.9 Initial Comment: I compiled Python from CVS this morning. It silently failed to compile ctypes. Here is the text surrounding the failure gcc [deleted] -c /Users/dalke/cvses/python-svn/Modules/_ctypes/ libffi/src/powerpc/darwin_closure.S -o build/temp.darwin-7.9.0- Power_Macintosh-2.5/darwin_closure.o darwin_closure.S:249:unknown section attribute: live_support darwin_closure.S:249:Rest of line ignored. 1st junk character valued 69 (E). building 'itertools' extension ... Python installed but when I tried to import ctypes I got File "/usr/local/lib/python2.5/ctypes/__init__.py", line 8, in from _ctypes import Union, Structure, Array ImportError: No module named _ctypes I tracked it down to the '+live_support' attribute from the darwin_closure.S. My compiler does not understand that. Thomas Heller (in private email) pointed out the text from the ctypes README On OS X, the segment attribute live_support must be defined. If your compiler doesn't know about it, upgrade or set the environment variable CCASFLAGS="-Dno_live_support". Upgrading is out of the option. I set the environment variable but that did not fix things when I tried to recompile Python. However, editing the file to remove the "+live_support" works. All the self-tests passed, and my experimentation this afternoon was successful. -- >Comment By: Andrew Dalke (dalke) Date: 2006-04-13 18:51 Message: Logged In: YES user_id=190903 Sorry - was ill and not doing anything for a week. I've built the latest code from SVN. No problems with the compilation and I am able to import just fine. I did get the following compiler warnings /Users/dalke/cvses/python-svn/Modules/_ctypes/cfield.c: In function `CField_repr': /Users/dalke/cvses/python-svn/Modules/_ctypes/cfield.c:259: warning: signed size_t format, Py_ssize_t arg (arg 3) /Users/dalke/cvses/python-svn/Modules/_ctypes/cfield.c:267: warning: signed size_t format, Py_ssize_t arg (arg 3) /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c:383: warning: function declaration isn't a prototype /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c:384: warning: function declaration isn't a prototype /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c:388: warning: function declaration isn't a prototype /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c:389: warning: function declaration isn't a prototype /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c:394: warning: function declaration isn't a prototype /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c: In function `ffi_closure_helper_DARWIN': /Users/dalke/cvses/python-svn/Modules/_ctypes/libffi/src/powerpc/ ffi_darwin.c:622: warning: unused variable `temp_ld' When I run the self test I get one error FAILED (errors=1) Traceback (most recent call last): File "Lib/test/test_ctypes.py", line 12, in test_main() File "Lib/test/test_ctypes.py", line 9, in test_main run_suite(unittest.TestSuite(suites)) File "/Users/dalke/cvses/python-svn/Lib/test/test_support.py", line 285, in run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "/Users/dalke/cvses/python-svn/Lib/ctypes/test/test_loading.py", line 30, in test_load cdll.load(libc_name) File "/Users/dalke/cvses/python-svn/Lib/ctypes/_loader.py", line 112, in load return self._load(libname, mode) File "/Users/dalke/cvses/python-svn/Lib/ctypes/_loader.py", line 153, in _load return self.load_library(pathname, mode) File "/Users/dalke/cvses/python-svn/Lib/ctypes/_loader.py", line 124, in load_library return self._dlltype(libname, mode) File "/Users/dalke/cvses/python-svn/Lib/ctypes/__init__.py", line 288, in __init__ self._handle = _dlopen(self._name, mode) OSError: dlcompat: unable to open this file with RTLD_LOCAL which is probably ignorable - OS X has its own sense of what shared libraries do. -- Comment By: Thomas Heller (theller) Date: 2006-04-03 14:18 Message: Logged In: YES user_id=11105 As a temporary fix, I removed the '+live_support' attribute in the
[ python-Bugs-1470212 ] mailbox.PortableUnixMailbox fails to parse 'From ' in body
Bugs item #1470212, was opened at 2006-04-14 03:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1470212&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Lars Kellogg-Stedman (larsks) Assigned to: Nobody/Anonymous (nobody) Summary: mailbox.PortableUnixMailbox fails to parse 'From ' in body Initial Comment: I have a Unix mailbox file that contains the following text in the body of a message: ---[ cut here ]--- EFCO also offers casements with integral blinds. See:=20 http://www.efcocorp.com/public/earm/products/default.asp?j=3D1&P=3D43&L=3D= 1=20 >From that page, select select "Features and Benefits" from under the heading "Product Overview"=20 ---[ cut here ]--- mailbox.PortableUnixMailbox erroneously interprets the "From" at the beginning of the line as the beginning of a new message. Since 'From ' is only a valid header at the beginning of a message, perhaps the module could look at the following line and see if it looks like an RFC2822 header before accepting 'From ' as a message delimiter. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1470212&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
