New submission from Evan Jones: The system version of libsqlite3 that is included in Mac OS X is not fork safe. This means that if a process forks, and the child calls into it, it will crash with the stack trace below.
I've reproduced this with both Python 2.7.10 and Python 3.5.1 on Mac OS X 10.11.5. There are a number of reports about this issue on the Internet. The only way I can think to solve this problem is to bundle SQLite with the Python source code, and build an included version. This will avoid the problem, since only Apple's fork of SQLite uses the problematic libdispatch library. Details: * Apple ships a version of sqlite3 that uses their "Grand Central Dispatch" libdispatch library. * Grand Central Dispatch is explicitly *not* thread safe. From their docs: https://developer.apple.com/library/mac/documentation/Performance/Reference/GCD_libdispatch_Ref/index.html "Be careful when mixing GCD with the fork system call. If a process makes GCD calls prior to calling fork, it is not safe to make additional GCD calls in the resulting child process until after a successful call to exec or related functions." * Some System APIs also seem to call into this library. In my case: urllib/urllib2 access the system's proxy settings. Related bugs: * I believe this is the root cause of https://bugs.python.org/issue20353 * Celery also has a detailed bug report: https://github.com/celery/celery/issues/869 * Pure native code can cause this crash: http://ludovicrousseau.blogspot.com/2015/01/os-x-yosemite-bug-pcsc-functions-crash.html Crash details from Mac OS X's system Python: Application Specific Information: crashed on child side of fork pre-exec Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libdispatch.dylib 0x00007fff92e8b162 _dispatch_barrier_async_f_slow + 356 1 libsqlite3.dylib 0x00007fff8a607656 sqlite3_initialize + 2950 2 libsqlite3.dylib 0x00007fff8a605b21 openDatabase + 65 3 _sqlite3.so 0x0000000105bb9a55 pysqlite_connection_init + 509 4 org.python.python 0x000000010567bc24 0x10562f000 + 314404 5 org.python.python 0x0000000105639202 PyObject_Call + 99 6 _sqlite3.so 0x0000000105bbdbf0 0x105bb8000 + 23536 7 org.python.python 0x00000001056b5a0b PyEval_EvalFrameEx + 13400 8 org.python.python 0x00000001056b8541 0x10562f000 + 562497 9 org.python.python 0x00000001056b530c PyEval_EvalFrameEx + 11609 10 org.python.python 0x00000001056b23c1 PyEval_EvalCodeEx + 1583 11 org.python.python 0x00000001056b1d8c PyEval_EvalCode + 54 12 org.python.python 0x00000001056d1a42 0x10562f000 + 666178 13 org.python.python 0x00000001056d1ae5 PyRun_FileExFlags + 133 14 org.python.python 0x00000001056d1634 PyRun_SimpleFileExFlags + 698 15 org.python.python 0x00000001056e3011 Py_Main + 3137 16 libdyld.dylib 0x00007fff8eb225ad start + 1 ---------- components: Macintosh files: osx_python_crash.py messages: 266396 nosy: evan.jo...@bluecore.com, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: Mac system sqlite3 not fork safe: Bundle a version? versions: Python 2.7, Python 3.5 Added file: http://bugs.python.org/file43003/osx_python_crash.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27126> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
