Christian Couder <christian.cou...@gmail.com> wrote: > Design discussion about performance > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Yeah, it is not efficient to fork/exec a command to just read or write > one object to or from the external ODB. Batch calls and/or using a > daemon and/or RPC should be used instead to be able to store regular > objects in an external ODB. But for now the external ODB would be all > about really big files, where the cost of a fork+exec should not > matter much. If we later want to extend usage of external ODBs, yeah > we will probably need to design other mechanisms.
I would also investigate switching run_command to use vfork+exec or posix_spawn for performance (keeping in mind vfork caveats documented at https://ewontfix.com/7/ ) posix_spawn in future glibc (probably 2.24) will use CLONE_VFORK in all cases under Linux, and posix_spawn may help with portability, too. I think the only thing we can't support with posix_spawn which run_command supports is chdir; all the redirections/closing FDs should be fine. With only 10MB malloc-ed, the following shows vfork performance being noticeably faster than plain fork: /* gcc -o vfork-test -Wall -O2 vfork-test.c */ #include <sys/types.h> #include <sys/wait.h> #include <unistd.h> #include <stdlib.h> #include <string.h> int main(int argc, char *argv[], char *envp[]) { int i; int do_vfork = argc > 1 && !strcmp(argv[1], "vfork"); char * const cmd[] = { "/bin/true", 0 }; size_t n = 1024 * 1024 * 10; char *mem = malloc(n); memset(mem, 'a', n); /* make sure it's really allocated */ for (i = 0; i < 10000; i++) { pid_t pid = do_vfork ? vfork() : fork(); if (pid == 0) { execve(cmd[0], cmd, envp); write(2, "exec error\n", 11); _exit(1); } waitpid(pid, 0, 0); } return 0; } -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
