We have run into what we think to be more or less a bug in PyQt4, in QProcess, somewhat indirectly. QProcess sometimes hangsup in threaded PyQt applications, at least on Linux systems. The situation is the following: - A PyQt program is using two or more threads (say, python threads) - one of the threads is using a QProcess (whatever thread), while the other is using python (of course).
We think we have understood what is going on: Internally, QProcess forks then probably uses a kind of exec(). But the child forked process sometimes gets locked on the python GIL lock. Then the calling process waits for the child, and also gets hung on a select() call. Actually, when forking a multi-threaded process, only the forking thread is duplicated: so the child process has only 1 thread. But the locks, mutexes and other threading items have kept the state they had in their parent process: some are locked, but the other threads which would normally release them do not exist in the child. Here, the python GIL happens to be locked by another thread in the parent process, and the child process seems to try to lock the GIL too. We guess QProcess calls a virtual method in the forked process, which is overloaded by SIP/PyQt, which in turn calls the Python API, and finally gets hung trying to lock the GIL. We have never encountered this problem using PyQt3, maybe because QProcess or its PyQt binding may be implemented differently. This threading + fork problem is a known difficulty in fork(), explained in the linux manpages for instance. We have 2 possible solutions for this problem: 1. Using pthreads, it is possible to define handlers functions that can take care of locking/unlocking the needed locks whenever fork() is called, both in the parent and child processes: this is done using the pthread_atfork() function. Typically PyQt may define such handlers to take care of the GIL locking when fork() is used. However we don't know if such a mechanism exists on non-pthread systems (Windows?). We also don't really know if the problem exists on Windows; possibly not because Windows doesn't use fork() but probably rather a spawn equivalent (?). 2. The problem actually happens because the python API gets called from the forked process. Normally, in C++ Qt, the QProcess doesn't trigger such a locking problem. However the SIP binding defines a subclass which traps all virtual methods calls and redirects them to python. So if we are not using a sip-inherited QProcess class, the problem should not happen. We have made the test, making a sip-bound factory function which instantiates a QProcess from C++, then returns it to the pyton layer. This way no inherited class is used, and the problem is avoided. And it actually works. Would it be difficult to have solution 1 included in the standard PyQt ? Denis _______________________________________________ PyQt mailing list PyQt@riverbankcomputing.com http://www.riverbankcomputing.com/mailman/listinfo/pyqt
