STINNER Victor <vstin...@redhat.com> added the comment:
"posix_spawn() is faster" This assumption needs a benchmark! I ran a benchmark on Linux (Fedora 29, Linux kernel 4.19.9-300.fc29.x86_64, glibc 2.28) using attached subprocess_bench.py. I rebased the PR 11242 on master and run the benchmark in virtual environment: git co pr/11242 # current branch: PR 11242 make distclean ./configure make ./python -m venv env env/bin/python -m pip install perf env/bin/python subprocess_bench.py -v -o posix_spawn.json git co master env/bin/python subprocess_bench.py -v -o fork_exec.json env/bin/python -m perf compare_to fork_exec.json posix_spawn.json The result is quite explicit: the PR makes subprocess.Popen 61x faster!!! $ env/bin/python -m perf compare_to fork_exec.json posix_spawn.json Mean +- std dev: [fork_exec] 27.1 ms +- 0.4 ms -> [posix_spawn] 447 us +- 163 us: 60.55x faster (-98%) That's the best case: * The parent process (Python) allocated 2 GiB of memory: that's not uncommon for large application. On OpenStack for example, it's common that a process takes more than 1 GiB. * The child process has a short execution time and allocates few memory. On my config (Fedora 29, Linux kernel 4.19.9-300.fc29.x86_64, glibc 2.28), os.posix_spawn() uses vfork(): $ strace -o trace ./python -c 'import subprocess; subprocess.run(["/bin/true"], close_fds=False, restore_signals=False)' $ grep clone trace clone(child_stack=0x7fab28ac0ff0, flags=CLONE_VM|CLONE_VFORK|SIGCHLD) = 23073 I guess that the 61x speedup mostly comes from vfork(). See also bpo-34663 for previous discussion about vfork() and os.posix_spawn(). In short, the glibc is smart and detects when vfork() can be used or not. ---------- Added file: https://bugs.python.org/file48011/subprocess_bench.py _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue35537> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
