Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-09 Thread Junio C Hamano
Jeff King writes: > Here's the code to do the cycle-breaking. Aside from the "hacky" bit, > it's quite simple. I added a new state enum to object_entry to handle > the graph traversal. Since it only needs 2 bits, I _assume_ a compiler > can fit it in with the bitfields above (or at the very leas

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-09 Thread Junio C Hamano
Jeff King writes: > I ran the repack again with stock git (no MRU patch), and the number of > objects in the delta phase jumped to 3087880, around 56,000 more than > with the MRU patch. So that's probably where the magic "3%" is coming > from. By looking at the smaller packs first, we are more l

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-09 Thread Jeff King
On Tue, Aug 09, 2016 at 10:04:11AM -0400, Jeff King wrote: > Here's the code to do the cycle-breaking. > [...] > It seems to perform well, and it does break the cycles (the later check > during the write does not kick in, and we get no warnings). I didn't dig > into the fates of specific objects,

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-09 Thread Jeff King
On Mon, Aug 08, 2016 at 10:16:32AM -0700, Junio C Hamano wrote: > Jeff King writes: > > >> It worries me a lot to lose the warning unconditionally, though. > >> That's the (only) coal-mine canary that lets us notice a problem > >> when we actually start hitting that last-ditch cycle breaking too

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-08 Thread Junio C Hamano
Jeff King writes: >> It worries me a lot to lose the warning unconditionally, though. >> That's the (only) coal-mine canary that lets us notice a problem >> when we actually start hitting that last-ditch cycle breaking too >> often. > > The dedicated cycle-detection will lose that warning, too (i

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-08 Thread Jeff King
On Mon, Aug 08, 2016 at 09:28:29AM -0700, Junio C Hamano wrote: > Jeff King writes: > > > Here's a list of approaches I think we can use to fix this: > > > > 1. squelch the warning and ignore it. The downside here, besides not > > warning the user about true in-pack cycles, is that we hav

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-08 Thread Junio C Hamano
Jeff King writes: > Here's a list of approaches I think we can use to fix this: > > 1. squelch the warning and ignore it. The downside here, besides not > warning the user about true in-pack cycles, is that we have no > opportunity to actually find a new delta (because we realize the

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-08-08 Thread Jeff King
On Fri, Jul 29, 2016 at 08:02:51AM -0700, Junio C Hamano wrote: > > So it's possible that the resulting pack > > is not as small as it could be (i.e., we break the chain with a base > > object, but it's possible if we looked that we could have broken the > > chain by making a delta against an exis

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-07-29 Thread Junio C Hamano
Jeff King writes: > On Fri, Jul 29, 2016 at 12:15:24AM -0400, Jeff King wrote: > > But because this series switches the order of pack-lookup between > objects, it is possible for us to find a `B` which is a delta against > `A` in one pack, and then another copy of `A` which is a delta against > a

Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-07-28 Thread Jeff King
On Fri, Jul 29, 2016 at 12:15:24AM -0400, Jeff King wrote: > Note that this reordering does have a potential impact on > the final pack, as we store only a single "found" pack for > each object, even if it is present in multiple packs. In > principle, any copy is acceptable, as they all refer to t

[PATCH v2 7/7] pack-objects: use mru list when iterating over packs

2016-07-28 Thread Jeff King
In the original implementation of want_object_in_pack(), we always looked for the object in every pack, so the order did not matter for performance. As of the last few patches, however, we can now often break out of the loop early after finding the first instance, and avoid looking in the other pa