On Fri, Sep 9, 2016 at 10:27 PM, Steve D'Aprano <steve+pyt...@pearwood.info> wrote: > On Fri, 9 Sep 2016 07:28 pm, Chris Angelico wrote: > >> We don't >> have a problem with threading and multiprocessing having very similar >> APIs, do we? Yet they exist to solve distinctly different problems. > > Surely not? > > I would think that threading and multiprocessing are two distinct > implementations of the same problem: how to run two or more chunks of code > at the same time. > > In CPython we usually say "use threads for I/O bound tasks, processes for > CPU bound tasks" but that's only because of the GIL. One doesn't need such > a distinction in Jython or IronPython.
You also want to use processes if you need the ability to kill one of them externally, or track resource usage separately, or have independence of other process-wide features such as current working directory. So there are some problems (eg multi-user services) where the additional isolation is important. In contrast, you want to use threads if you need the ability to quickly and easily share mutable data, or if you want all resource usage to be lumped together - eg if you're not really doing several separate jobs, but are doing one single conceptual job. >From a systems administration POV, threads logically belong together, but processes are distinct beasts that communicate through clearly-defined IPC. There are times when you want one, and times when you want the other. The GIL just pulls a specific category of problem out of the hands of threads and into the hands of processes, due to its inability to spread Python code across CPU cores; but it didn't create the distinction. If a future version of CPython eliminates the GIL and allows threads to concurrently run CPU-heavy code, there will still be a need for multiprocessing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
