Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-31 Thread Jeff King
On Mon, Aug 31, 2015 at 11:56:04AM -0700, Junio C Hamano wrote: > Stefan Beller writes: > > > +static void destroy_output_mutex() > > static void destroy_output_mutex(void) Yep. Stefan, you may want to beef up the warning flags in your config.mak. For reference, I use: CFLAGS += -Wall -Werr

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-31 Thread Junio C Hamano
Stefan Beller writes: > +static void destroy_output_mutex() static void destroy_output_mutex(void) > +{ > + pthread_mutex_destroy(&output_mutex); > +} > + > +#else > +#define set_output_mutex() > +#define destroy_output_mutex() > +#define lock_output_mutex() > +#define unlock_output_mutex()

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Stefan Beller
On Fri, Aug 28, 2015 at 11:53 AM, Jeff King wrote: > On Fri, Aug 28, 2015 at 11:50:50AM -0700, Jonathan Nieder wrote: > >> > But what I meant was: the child will only show progress if stderr is a >> > tty, but here it is not. >> >> For clone / fetch, we can pass --progress explicitly. >> >> For so

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Stefan Beller
On Fri, Aug 28, 2015 at 11:44 AM, Jeff King wrote: > On Fri, Aug 28, 2015 at 11:41:17AM -0700, Stefan Beller wrote: > >> > So to an observer, it would look like a serial operation, but subsequent >> > operations after the first would magically go much faster (because >> > they'd been working and b

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jeff King
On Fri, Aug 28, 2015 at 11:50:50AM -0700, Jonathan Nieder wrote: > > But what I meant was: the child will only show progress if stderr is a > > tty, but here it is not. > > For clone / fetch, we can pass --progress explicitly. > > For some reason 'git checkout' doesn't support a --progress optio

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jonathan Nieder
Jeff King wrote: > Right, clearly we can't rely on pipe buffers to be large enough here > (though we _may_ want to rely on tempfiles if we aren't sure that the > stdout is bounded in a reasonable way). > > But what I meant was: the child will only show progress if stderr is a > tty, but here it is

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jonathan Nieder
Jeff King wrote: > I think Jonathan's point is that you could pick _one_ active child to > show without buffering, while simultaneously buffering everybody else's > output. Yep. Thanks for interpreting. [...] > So to an observer, it would look like a serial operation, but subsequent > operation

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jeff King
On Fri, Aug 28, 2015 at 11:41:17AM -0700, Stefan Beller wrote: > > So to an observer, it would look like a serial operation, but subsequent > > operations after the first would magically go much faster (because > > they'd been working and buffering in the background). > > > > And that doesn't requ

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jonathan Nieder
Junio C Hamano wrote: > Jonathan Nieder writes: >> remote: Finding sources: 11% (18/155) \r >> remote: Finding sources: 12% (19/155) \r >> >> With multiple commands producing such output, they will overwrite each >> other's lines, producing a mixture that is confusing and

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Stefan Beller
On Fri, Aug 28, 2015 at 11:35 AM, Jeff King wrote: > On Fri, Aug 28, 2015 at 11:27:04AM -0700, Junio C Hamano wrote: > >> > But for commands that show progress like "git clone", "git checkout", >> > and "git fetch", it does not work well at all. They provide output >> > that updates itself by put

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Junio C Hamano
Jeff King writes: > I think Jonathan's point is that you could pick _one_ active child to > show without buffering, while simultaneously buffering everybody else's > output. When that finishes, pick a new active child, show its buffer, > and then start showing its output in realtime. And so on. >

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Stefan Beller
On Fri, Aug 28, 2015 at 11:27 AM, Junio C Hamano wrote: > Jonathan Nieder writes: > >> Junio C Hamano wrote: >> >>> My preference is still (1) leave standard error output all connected >>> to the same fd without multiplexing, and (2) line buffer standard >>> output so that the output is at least

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jeff King
On Fri, Aug 28, 2015 at 11:27:04AM -0700, Junio C Hamano wrote: > > But for commands that show progress like "git clone", "git checkout", > > and "git fetch", it does not work well at all. They provide output > > that updates itself by putting a carriage return at the end of each > > chunk of out

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Junio C Hamano
Jonathan Nieder writes: > Junio C Hamano wrote: > >> My preference is still (1) leave standard error output all connected >> to the same fd without multiplexing, and (2) line buffer standard >> output so that the output is at least readable as a text, in a >> similar way a log of an irc channel w

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jonathan Nieder
Junio C Hamano wrote: > My preference is still (1) leave standard error output all connected > to the same fd without multiplexing, and (2) line buffer standard > output so that the output is at least readable as a text, in a > similar way a log of an irc channel where everybody is talking at > th

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Stefan Beller
On Fri, Aug 28, 2015 at 10:12 AM, Junio C Hamano wrote: > Jonathan Nieder writes: > >> Stefan Beller wrote: >>> On Thu, Aug 27, 2015 at 6:14 PM, Stefan Beller wrote: >> This makes use of the new task queue and the syncing feature of run-command to fetch a number of submodules at the sa

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Junio C Hamano
Jonathan Nieder writes: > Stefan Beller wrote: >> On Thu, Aug 27, 2015 at 6:14 PM, Stefan Beller wrote: > >>> This makes use of the new task queue and the syncing feature of >>> run-command to fetch a number of submodules at the same time. >>> >>> The output will look like it would have been run

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Jonathan Nieder
Stefan Beller wrote: > On Thu, Aug 27, 2015 at 6:14 PM, Stefan Beller wrote: >> This makes use of the new task queue and the syncing feature of >> run-command to fetch a number of submodules at the same time. >> >> The output will look like it would have been run sequential, >> but faster. > > An

Re: [PATCH 7/9] fetch: fetch submodules in parallel

2015-08-28 Thread Stefan Beller
On Thu, Aug 27, 2015 at 6:14 PM, Stefan Beller wrote: > This makes use of the new task queue and the syncing feature of > run-command to fetch a number of submodules at the same time. > > The output will look like it would have been run sequential, > but faster. And it breaks the tests t5526-fetc

[PATCH 7/9] fetch: fetch submodules in parallel

2015-08-27 Thread Stefan Beller
This makes use of the new task queue and the syncing feature of run-command to fetch a number of submodules at the same time. The output will look like it would have been run sequential, but faster. Signed-off-by: Stefan Beller --- Documentation/fetch-options.txt | 7 +++ builtin/fetch.c