On 7 Sep, 14:50, MRAB <pyt...@mrabarnett.plus.com> wrote: > CPython's GIL means that multithreading on multiple processors/cores has > limitations. Each interpreter has its own GIL, so processor-intensive > applications work better using the multiprocessing module than with the > threading module.
We incur a 200x speed-penalty from Python if the code is CPU-bound. How much do you gain from one extra processor? Start by adding in some C, and you will gain much more performance, even without parallel processing. Processor-intensive code should therefore use extension libraries that release the GIL. Then you have option of getting parallel concurrency by Python threads or OpenMP in C/Fortran. Most processor-intensive code depend on numerical libraries written in C or Fortran anyway (NumPy, ATLAS, LAPACK, FFTW, GSL, MKL, etc.) When you do this, the GIL does not get in your way. The GIL is infact an advantage if one library happen not to be thread safe. Many old Fortran 77 libraries have functions with re-entrancy issues, due to SAVE attribute on variables. Thus, the GIL is often an advantage for processor-intensive code. multiprocessing is fragile and unreliable, btw. -- http://mail.python.org/mailman/listinfo/python-list
