[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Jesse Noller <[EMAIL PROTECTED]> added the comment: I've committed this as-is based off my last patch. I will watch the buildbots for failures. Mark/Amaury - if I see you guys at pycon, I owe you a drink. -- resolution: -> fixed status: open -> closed ___

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Jesse Noller <[EMAIL PROTECTED]> added the comment: I added the semicolons Amaury, and have it teed up in my local repo for submit. Can you review this diff just to confirm? Added file: http://bugs.python.org/file11031/add_semicolons.diff ___ Python tracker

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: Mark, There are 3 semicolons missing in your patch, in pipe_connection.c, just after the calls to WriteFile and ReadFile. After this, it compiles and runs correctly. Tests pass. Note that on Windows, your "nine lines" cannot work as is,

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: Thanks, Amaury! How's this? I have no access to a Windows machine, so this patch is untested on Windows. Added file: http://bugs.python.org/file11028/issue3399_2.patch ___ Python tracker <[EMAIL PROTECT

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: To be complete, the patch should also deal with conn_recv_string() which has the same problem. And please do not forget the win32 implementation, in pipe_connection.c. ___ Python tracker <[EMAIL PROTE

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: Here's a patch that fixes the problem for me. It releases the GIL around the calls to _conn_sendall within conn_send_string, instead of releasing the GIL for the whole call to conn_send_string. -- keywords: +patch Added file: http:/

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: > This is wrong (the GIL must be held when using the PyMem_* and > PyObject_* functions), and is probably the cause of the failed assertion. This sounds quite likely. I just managed (using the low-tech method of setting a static variable o

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment: > Assertion failed: (bp != NULL), function PyObject_Malloc, file > Objects/obmalloc.c, line 755. This one gives one probable cause of the problem: - in Modules/_multiprocessing/connection.h, connection_send_obj() releases the GIL arou

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: I finally found some more time to look at this. I cut down the test-suite to try to find a minimal failing example. I can fairly reliably make a debug build of the trunk crash using the following nine lines import multiprocessing.managers

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Jesse Noller <[EMAIL PROTECTED]> added the comment: Ok, so for the moment, let's set aside the connection refused messages: that may be a case of not cleaning up a socket correctly (which is still bad, but not memory corruption). Of note from the buildbot failure: Assertion failed: (bp != NULL

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: It looks like this isn't just me. See the buildbot output at: http://www.python.org/dev/buildbot/all/x86%20osx.5%20trunk/builds/33/ste p-test/0 which shows: test_multiprocessing Assertion failed: (bp != NULL), function PyObject_Malloc, fil

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: I should add to the previous message that this was revision 65090, and that it was a non-debug build. ___ Python tracker <[EMAIL PROTECTED]> __

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: Okay: I just tried the following: (1) clean svn checkout (2) ./configure && make (3) 100 runs of test_multiprocessing, via the shell command: for ((i=0;i<100;i+=1)); do ./python.exe Lib/test/test_multiprocessing.py; sleep 1; done I got 4 f

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Jesse Noller <[EMAIL PROTECTED]> added the comment: Can you try removing the --with-pydebug flag from configure and running that way? ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: LD_LIBRARY_PATH isn't set. gcc is the system gcc from Apple: Macintosh-3:trunk dickinsm$ echo $LD_LIBRARY_PATH Macintosh-3:trunk dickinsm$ which gcc /usr/bin/gcc Macintosh-3:trunk dickinsm$ gcc -v Using built-in specs. Target: i686-apple-da

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Jesse Noller <[EMAIL PROTECTED]> added the comment: On Jul 17, 2008, at 6:22 PM, Mark Dickinson <[EMAIL PROTECTED]> wrote: > > New submission from Mark Dickinson <[EMAIL PROTECTED]>: > > As of revision 65077 of the trunk, I'm getting errors in > test_multiprocessing that seem to point to memor

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: And another: Failure 5: test_notify (__main__.WithManagerTestCondition) ... Assertion failed: (usable_arenas->freepools == NULL), function PyObject_Malloc, file Objects/obmalloc.c, line 809. ERROR ___ P

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

Mark Dickinson <[EMAIL PROTECTED]> added the comment: And one more: Failure 4: test_make_pool (__main__.WithManagerTestPool) ... Assertion failed: (bp != NULL), function PyObject_Malloc, file Objects/obmalloc.c, line 746. ___ Python tracker <[EMAIL PROTECTE

[issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

New submission from Mark Dickinson <[EMAIL PROTECTED]>: As of revision 65077 of the trunk, I'm getting errors in test_multiprocessing that seem to point to memory corruption in object allocation/deallocation. The failures are intermittent, and of a similar nature to the errors I was seeing pr