Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Ævar Arnfjörð Bjarmason
On Fri, May 24 2019, SZEDER Gábor wrote: > On Fri, May 24, 2019 at 01:58:03PM +0200, Ævar Arnfjörð Bjarmason wrote: >> >> On Fri, May 24 2019, SZEDER Gábor wrote: >> >> > On Fri, May 24, 2019 at 01:17:06PM +0200, Ævar Arnfjörð Bjarmason wrote: >> >> >> >> On Fri, May 24 2019, SZEDER Gábor wrote:

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread SZEDER Gábor
On Fri, May 24, 2019 at 01:58:03PM +0200, Ævar Arnfjörð Bjarmason wrote: > > On Fri, May 24 2019, SZEDER Gábor wrote: > > > On Fri, May 24, 2019 at 01:17:06PM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > >> On Fri, May 24 2019, SZEDER Gábor wrote: > >> > >> > On Fri, May 24, 2019 at 11:01:39AM +0

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Ævar Arnfjörð Bjarmason
On Fri, May 24 2019, SZEDER Gábor wrote: > On Fri, May 24, 2019 at 01:17:06PM +0200, Ævar Arnfjörð Bjarmason wrote: >> >> On Fri, May 24 2019, SZEDER Gábor wrote: >> >> > On Fri, May 24, 2019 at 11:01:39AM +0200, Ævar Arnfjörð Bjarmason wrote: >> >> I don't think it's a performance problem to ha

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread SZEDER Gábor
On Fri, May 24, 2019 at 01:17:06PM +0200, Ævar Arnfjörð Bjarmason wrote: > > On Fri, May 24 2019, SZEDER Gábor wrote: > > > On Fri, May 24, 2019 at 11:01:39AM +0200, Ævar Arnfjörð Bjarmason wrote: > >> I don't think it's a performance problem to have an old commit-graph > >> lying around. But if

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Ævar Arnfjörð Bjarmason
On Fri, May 24 2019, SZEDER Gábor wrote: > On Fri, May 24, 2019 at 11:01:39AM +0200, Ævar Arnfjörð Bjarmason wrote: >> I don't think it's a performance problem to have an old commit-graph >> lying around. But if you turn on the commit-graph, run gc a bunch, then >> turn it off in config we'll ha

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Derrick Stolee
On 5/24/2019 3:24 AM, Jeff King wrote: > On Thu, May 23, 2019 at 08:53:56AM -0400, Derrick Stolee wrote: > >>> I spent a while thinking and experimenting with this tonight. The result >>> is the patch below. Ævar, do you still have a copy of the repo that >>> misbehaved? I'd be curious to see how

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread SZEDER Gábor
On Fri, May 24, 2019 at 11:01:39AM +0200, Ævar Arnfjörð Bjarmason wrote: > I don't think it's a performance problem to have an old commit-graph > lying around. But if you turn on the commit-graph, run gc a bunch, then > turn it off in config we'll have it lying around forever, even if you do > subs

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Ævar Arnfjörð Bjarmason
On Fri, May 24 2019, Jeff King wrote: > On Fri, May 24, 2019 at 09:55:05AM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > I'm not sure what tickles the bitmap-writer to fail so hard. Is it >> > having too many refs? Weird patterns in the graph? Just a ton of >> > commits? >> >> Ah, why did only th

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Jeff King
On Fri, May 24, 2019 at 09:55:05AM +0200, Ævar Arnfjörð Bjarmason wrote: > > I'm not sure what tickles the bitmap-writer to fail so hard. Is it > > having too many refs? Weird patterns in the graph? Just a ton of > > commits? > > Ah, why did only this ancient (big) pack have a bitmap? > > The bi

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Ævar Arnfjörð Bjarmason
On Fri, May 24 2019, Jeff King wrote: > On Thu, May 23, 2019 at 09:26:34PM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > I spent a while thinking and experimenting with this tonight. The result >> > is the patch below. Ævar, do you still have a copy of the repo that >> > misbehaved? I'd be curiou

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Jeff King
On Thu, May 23, 2019 at 09:26:34PM +0200, Ævar Arnfjörð Bjarmason wrote: > > I spent a while thinking and experimenting with this tonight. The result > > is the patch below. Ævar, do you still have a copy of the repo that > > misbehaved? I'd be curious to see how it fares. > > No, sorry. I think

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Jeff King
On Thu, May 23, 2019 at 08:53:56AM -0400, Derrick Stolee wrote: > > I spent a while thinking and experimenting with this tonight. The result > > is the patch below. Ævar, do you still have a copy of the repo that > > misbehaved? I'd be curious to see how it fares. > > This patch caught my attenti

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-23 Thread Ævar Arnfjörð Bjarmason
On Thu, May 23 2019, Jeff King wrote: > On Wed, Apr 10, 2019 at 06:57:21PM -0400, Jeff King wrote: > >> 2. The answer we get from a bitmap versus a regular traversal are not >> apples-to-apples equivalent. The regular traversal walks down to >> the UNINTERESTING commits, marks the bo

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-23 Thread Derrick Stolee
On 5/23/2019 7:30 AM, Jeff King wrote: > On Wed, Apr 10, 2019 at 06:57:21PM -0400, Jeff King wrote: > >> 2. The answer we get from a bitmap versus a regular traversal are not >> apples-to-apples equivalent. The regular traversal walks down to >> the UNINTERESTING commits, marks the bou

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-23 Thread Jeff King
On Wed, Apr 10, 2019 at 06:57:21PM -0400, Jeff King wrote: > 2. The answer we get from a bitmap versus a regular traversal are not > apples-to-apples equivalent. The regular traversal walks down to > the UNINTERESTING commits, marks the boundaries trees and blobs as > UNINTERESTIN

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Junio C Hamano
Ævar Arnfjörð Bjarmason writes: > On Sat, May 04 2019, SZEDER Gábor wrote: > ... >> And don't forget that the commit-graph progress bar still prints >> nonsense :) > > I'm inclined to just wait until Derrick's refactorings land post-2.22.0 Fine by me. Thanks.

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Jeff King
On Wed, May 08, 2019 at 06:13:58PM +0200, Ævar Arnfjörð Bjarmason wrote: > > I do wonder if this really needs to be part of the progress bar. The > > goal of the progress bar is to give the user a sense that work is > > happening, and (if possible, but not for "enumerating") an idea of when > > it

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Ævar Arnfjörð Bjarmason
On Sat, May 04 2019, SZEDER Gábor wrote: > On Sat, May 04, 2019 at 08:52:01AM +0200, Ævar Arnfjörð Bjarmason wrote: >> As an aside this is the Nth time I notice how crappy that "Enumerating >> objects" progress bar is. > > And don't forget that the commit-graph progress bar still prints > nonsen

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Ævar Arnfjörð Bjarmason
On Wed, May 08 2019, Jeff King wrote: > On Tue, May 07, 2019 at 10:12:06AM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > I think we'd want a way to tell the bitmap code to update our progress >> > meter as it traverses (both single objects, but also taking into account >> > when it finds a bitmap

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Derrick Stolee
On 5/8/2019 3:11 AM, Jeff King wrote: > On Tue, May 07, 2019 at 10:12:06AM +0200, Ævar Arnfjörð Bjarmason wrote: > >>> I think we'd want a way to tell the bitmap code to update our progress >>> meter as it traverses (both single objects, but also taking into account >>> when it finds a bitmap and

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-08 Thread Jeff King
On Tue, May 07, 2019 at 10:12:06AM +0200, Ævar Arnfjörð Bjarmason wrote: > > I think we'd want a way to tell the bitmap code to update our progress > > meter as it traverses (both single objects, but also taking into account > > when it finds a bitmap and then suddenly bumps the value by a large >

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-07 Thread Ævar Arnfjörð Bjarmason
On Tue, May 07 2019, Jeff King wrote: > On Sat, May 04, 2019 at 08:52:01AM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > Note that Ævar's case was somebody running bitmaps locally and trying to >> > push, which I think is generally not a good match for bitmaps (even when >> > they work, they cost

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-07 Thread Jeff King
On Sat, May 04, 2019 at 08:52:01AM +0200, Ævar Arnfjörð Bjarmason wrote: > > Note that Ævar's case was somebody running bitmaps locally and trying to > > push, which I think is generally not a good match for bitmaps (even when > > they work, they cost more to generate than what you save if you're

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-04 Thread SZEDER Gábor
On Sat, May 04, 2019 at 08:52:01AM +0200, Ævar Arnfjörð Bjarmason wrote: > As an aside this is the Nth time I notice how crappy that "Enumerating > objects" progress bar is. And don't forget that the commit-graph progress bar still prints nonsense :) https://public-inbox.org/git/87ef6ydds8@

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-03 Thread Ævar Arnfjörð Bjarmason
On Sat, May 04 2019, Jeff King wrote: > On Thu, Apr 25, 2019 at 04:16:46PM +0900, Junio C Hamano wrote: > >> I was revisiting the recent "What's cooking" report, and I am not >> sure what the current status of the topic is. >> >> I do not get a feel that the current bitmap implementation has bee

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-03 Thread Jeff King
On Thu, Apr 25, 2019 at 04:16:46PM +0900, Junio C Hamano wrote: > I was revisiting the recent "What's cooking" report, and I am not > sure what the current status of the topic is. > > I do not get a feel that the current bitmap implementation has been > widely used in repositories that have vastl

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-04-25 Thread Junio C Hamano
Jeff King writes: > On Tue, Apr 09, 2019 at 05:10:43PM +0200, Ævar Arnfjörð Bjarmason wrote: > >> I've found a case where turning bitmaps on does horrible things for >> bitmap "push" performance. >> [...] >> I can't share the repo, but I had a report where just a "git push" of a >> topic branch t

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-04-10 Thread Jeff King
On Tue, Apr 09, 2019 at 05:10:43PM +0200, Ævar Arnfjörð Bjarmason wrote: > I've found a case where turning bitmaps on does horrible things for > bitmap "push" performance. > [...] > I can't share the repo, but I had a report where just a "git push" of a > topic branch that was 2/58 ahead/behind to

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-04-09 Thread Ævar Arnfjörð Bjarmason
On Thu, Mar 14 2019, Eric Wong wrote: > Jeff King wrote: >> On Wed, Mar 13, 2019 at 01:51:33AM +, Eric Wong wrote: >> >> > But I did find Ævar's forgotten gitperformance doc and thread >> > where the topic was brought up: >> > >> > https://public-inbox.org/git/20170403211644.26814-1-ava..

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-03-14 Thread Jeff King
On Thu, Mar 14, 2019 at 09:12:54AM +, Eric Wong wrote: > > The reason it defaults to off is for on-disk compatibility with JGit. > > Right. Our documentation seems to indicate JGit just warns (but > doesn't fall over), so maybe that can be considered separately. I think it was a hard error

[PATCH v3] repack: enable bitmaps by default on bare repos

2019-03-14 Thread Eric Wong
Jeff King wrote: > On Wed, Mar 13, 2019 at 01:51:33AM +, Eric Wong wrote: > > > But I did find Ævar's forgotten gitperformance doc and thread > > where the topic was brought up: > > > > https://public-inbox.org/git/20170403211644.26814-1-ava...@gmail.com/ > > One thing that thread reminde