Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-14 Thread Jeff King
On Mon, Sep 14, 2015 at 10:55:09AM -0700, Jonathan Nieder wrote: > > I don't think you need exact timing information. This is no different > > than running the commands themselves, with stdout and stderr writing to > > a pty that your terminal emulator will then read() from. If the program > > pr

Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-14 Thread Junio C Hamano
Jeff King writes: > I don't think you need exact timing information. This is no different > than running the commands themselves, with stdout and stderr writing to > a pty that your terminal emulator will then read() from. If the program > produces intermingled stdout/stderr that clogs up the te

Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-14 Thread Jonathan Nieder
Jeff King wrote: > On Mon, Sep 14, 2015 at 09:46:58AM -0700, Stefan Beller wrote: >> I tried implementing a buffering solution for both stdout and stderr, >> but that doesn't really workout well if you consider interleaved output >> on the pipes as we cannot accurately replay that later on. [...]

Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-14 Thread Stefan Beller
On Mon, Sep 14, 2015 at 10:17 AM, Jeff King wrote: > On Mon, Sep 14, 2015 at 09:46:58AM -0700, Stefan Beller wrote: > >> I tried implementing a buffering solution for both stdout and stderr, >> but that doesn't really workout well if you consider interleaved output >> on the pipes as we cannot acc

Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-14 Thread Jeff King
On Mon, Sep 14, 2015 at 09:46:58AM -0700, Stefan Beller wrote: > I tried implementing a buffering solution for both stdout and stderr, > but that doesn't really workout well if you consider interleaved output > on the pipes as we cannot accurately replay that later on. To do that > we would need t

Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-14 Thread Stefan Beller
On Sat, Sep 12, 2015 at 12:11 PM, Junio C Hamano wrote: >> + if (start_command(&children[i])) >> + die(_("Could not start child process")); >> + flags = fcntl(children[i].err, F_GETFL); >> + fcntl(children[i].e

Re: [PATCH 2/2] fetch: fetch submodules in parallel

2015-09-12 Thread Junio C Hamano
Stefan Beller writes: > diff --git a/run-command.c b/run-command.c > index 28e1d55..b8ff67b 100644 > --- a/run-command.c > +++ b/run-command.c > @@ -852,3 +852,147 @@ int capture_command(struct child_process *cmd, struct > strbuf *buf, size_t hint) > close(cmd->out); > return finish_

[PATCH 2/2] fetch: fetch submodules in parallel

2015-09-11 Thread Stefan Beller
If we run external commands in parallel we cannot pipe the output directly to the our stdout/err as it would mix up. So each process's output will flow through a pipe, which we buffer. One subprocess can be directly piped to out stdout/err for a low latency feedback to the user. Example: Let's ass