On Fri, Aug 25, 2017 at 3:40 AM, Marko Rauhamaa <ma...@pacujo.net> wrote: > Chris Angelico <ros...@gmail.com>: > >> On Fri, Aug 25, 2017 at 12:17 AM, Jon Ribbens >> <jon+use...@unequivocal.eu> wrote: >>> By that, do you mean "kill the process"? That's obviously not a >>> sensible answer in general, especially given we were including >>> processes which have no terminal or user sitting there watching them. >> >> Only in the sense that "kill" is the Unix term for "send signal". >> Python catches the signal, the system call terminates with EINTR, and >> the exception is raised. Give it a try. > > Signals are an arcane Unix communication method. I strongly recommend > against using signals for anything but terminating a process, and even > then you have to be extra careful. > > I have seen code that uses signals for runtime communication, but none > of it was free from race conditions.
Strongly disagree. Signals exist so that they can be used. Sending SIGHUP to a daemon to tell it to reload its configs is well-supported by the ecosystem; use of SIGCHLD and SIGWINCH for non-termination conditions is also vital. How else should an operating system or desktop environment inform a process of something important? Ctrl-C sends SIGINT meaning "interrupt". There are many, MANY situations in which "interrupting" a process doesn't terminate it. Here's one very simple example: $ python3 Python 3.7.0a0 (heads/master:3913bad495, Jul 21 2017, 20:53:52) [GCC 6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ( ... KeyboardInterrupt >>> You hit Ctrl-C in the middle of typing a multi-line command (maybe you didn't expect it to be multi-line?), and the current input is cancelled. Easy. No termination needed. And no race condition. I don't know why you'd get those, but they're not hard to eliminate. >>>>> Basically the only way to use requests safely is to fork an >>>>> individual process for every request, which is of course >>>>> spectacularly inefficient. >>>> >>>> Yes. Spectacularly inefficient and almost certainly unnecessary. > > Processes are a nice way to exercise multiple CPUs and also a way to > deal with obnoxious synchronous function calls. > > However, you don't want to fork a process (or a thread, for that matter) > for each context. Rather, you should have a pool of processes in the > order of, say, 2 * CPU count, and the processes should fetch work from a > queue. > > And if a process should get stuck, killing it is trivial. Yeah, if you DO need to split them out, a pool is good. I didn't bother mentioning it as a single process was enough for the scenario in question (which definitely sounded like an I/O-bound app), but even if you do fork, there's not a lot of point forking exactly one per request. Interestingly, the 2*CPUs figure isn't always optimal. I've messed around with -j options on repeatable tasks, and sometimes the best figure is lower than that, and other times it's insanely higher. On my four-core-hyperthreading CPU, I've had times when 12 is right, I've had times when 4 is right, and I even had one situation when I was debating between -j64 and -j128 on a task that was theoretically CPU-bound (ray-tracing). There are a lot of weird things happening with caching and such, and the only way to truly know what's best is to try it! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
