Re: Is offloading to GPU a worthwhile feature?

2018-04-11 Thread Jakub Narebski
Konstantin Ryabitsev writes: > On 04/08/18 09:59, Jakub Narebski wrote: >>> This is an entirely idle pondering kind of question, but I wanted to >>> ask. I recently discovered that some edge providers are starting to >>> offer systems with GPU cards in them -- p

Re: [PATCH 0/6] Compute and consume generation numbers

2018-04-11 Thread Jakub Narebski
Derrick Stolee writes: > On 4/7/2018 12:55 PM, Jakub Narebski wrote: >> Currently I am at the stage of reproducing results in FELINE paper: >> "Reachability Queries in Very Large Graphs: A Fast Refined Online Search >> Approach" by Renê R. Veloso, Loïc Cerf, Wa

Re: [PATCH v2 0/4] Lazy-load trees when reading commit-graph

2018-04-11 Thread Jakub Narebski
Derrick Stolee writes: > On 4/7/2018 2:40 PM, Jakub Narebski wrote: >> Derrick Stolee writes: >> >> [...] >>> On the Linux repository, performance tests were run for the following >>> command: >>> >>> git log --graph --onelin

Re: What's cooking in git.git (Apr 2018, #01; Mon, 9)

2018-04-11 Thread Jakub Narebski
Junio C Hamano writes: > * ab/nuke-emacs-contrib (2018-03-13) 1 commit > (merged to 'next' on 2018-03-15 at 13eb4e2d8b) > + git{,-blame}.el: remove old bitrotting Emacs code > > The scripts in contrib/emacs/ have outlived their usefulness and > have been removed. > > Will kick back to 'pu'

Re: [PATCH v8 03/14] commit-graph: add format document

2018-04-11 Thread Jakub Narebski
Derrick Stolee writes: > +CHUNK DATA: > + > + OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes) > + The ith entry, F[i], stores the number of OIDs with first > + byte at most i. Thus F[255] stores the total > + number of commits (N). > + > + OID Lookup (ID: {'O', 'I', 'D', 'L

Re: File versioning based on shallow Git repositories?

2018-04-13 Thread Jakub Narebski
Hallvard Breien Furuseth writes: > Also maybe it'll be worthwhile to generate .git/info/grafts in a local > clone of the repo to get back easily visible history. No grafts in > the original repo, grafts mess things up. Just a reminder: modern Git has "git replace", a modern and safe alternative

Re: [PATCH v2 07/10] commit-graph.txt: update future work

2018-04-13 Thread Jakub Narebski
Derrick Stolee writes: > On 4/12/2018 5:12 AM, Junio C Hamano wrote: >> Derrick Stolee writes: >> >>> +Here is a diagram to visualize the shape of the full commit graph, and >>> +how different generation numbers relate: >>> + >>> ++-+ >>> +| GENERA

Re: [PATCH] completion: reduce overhead of clearing cached --options

2018-04-13 Thread Jakub Narebski
SZEDER Gábor writes: > To get the names of all '$__git_builtin_*' variables caching --options > of builtin commands in order to unset them, 8b0eaa41f2 (completion: > clear cached --options when sourcing the completion script, > 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash

Re: File versioning based on shallow Git repositories?

2018-04-13 Thread Jakub Narebski
Hello Johannes, Johannes Schindelin writes: > On Fri, 13 Apr 2018, Jakub Narebski wrote: >> Hallvard Breien Furuseth writes: >> >>> Also maybe it'll be worthwhile to generate .git/info/grafts in a local >>> clone of the repo to get back easily visible h

Re: [PATCH v8 03/14] commit-graph: add format document

2018-04-13 Thread Jakub Narebski
Derrick Stolee writes: > On 4/11/2018 4:58 PM, Jakub Narebski wrote: >> Derrick Stolee writes: >> >>> +CHUNK DATA: >>> + >>> + OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes) >>> + The ith entr

Re: [PATCH] completion: reduce overhead of clearing cached --options

2018-04-14 Thread Jakub Narebski
SZEDER Gábor writes: > On Fri, Apr 13, 2018 at 11:44 PM, Jakub Narebski wrote: >> SZEDER Gábor writes: >>> >>> In Bash we can do better: run the 'compgen -v __gitcomp_builtin_' >>> builtin command, which lists the same variables, but without a

Re: [PATCH 0/6] Compute and consume generation numbers

2018-04-14 Thread Jakub Narebski
Derrick Stolee writes: > On 4/11/2018 3:32 PM, Jakub Narebski wrote: >> What would you suggest as a good test that could imply performance? The >> Google Colab notebook linked to above includes a function to count >> number of commits (nodes / vertices in the commit graph)

Re: [PATCH v8 09/14] commit-graph: add core.commitGraph setting

2018-04-14 Thread Jakub Narebski
Derrick Stolee writes: > From: Derrick Stolee > > The commit graph feature is controlled by the new core.commitGraph config > setting. This defaults to 0, so the feature is opt-in. > > The intention of core.commitGraph is that a user can always stop checking > for or parsing commit graph files i

Re: [PATCH v8 08/14] commit-graph: implement git commit-graph read

2018-04-14 Thread Jakub Narebski
Derrick Stolee writes: > From: Derrick Stolee > Subject: [PATCH v8 08/14] commit-graph: implement git commit-graph read Minor nit: this is one commit message [subject] among all others that uses "git commit-graph" instead of "git-commit-graph" in the description. Also, perhaps this (and simila

Re: [PATCH v8 04/14] graph: add commit graph design document

2018-04-15 Thread Jakub Narebski
Derrick Stolee writes: > +Future Work > +--- > + > +- The commit graph feature currently does not honor commit grafts. This can > + be remedied by duplicating or refactoring the current graft logic. The problem in my opinion lies in different direction, namely that commit grafts can cha

Re: [PATCH v3 3/9] commit: use generations in paint_down_to_common()

2018-04-18 Thread Jakub Narebski
Derrick Stolee writes: > Define compare_commits_by_gen_then_commit_date(), which uses generation > numbers as a primary comparison and commit date to break ties (or as a > comparison when both commits do not have computed generation numbers). > > Since the commit-graph file is closed under reacha

Re: [PATCH v3 4/9] commit-graph.txt: update design document

2018-04-18 Thread Jakub Narebski
Derrick Stolee writes: > We now calculate generation numbers in the commit-graph file and use > them in paint_down_to_common(). All right. > > Expand the section on generation numbers to discuss how the three > special generation numbers GENERATION_NUMBER_INFINITY, _ZERO, and > _MAX interact wi

Re: [PATCH v3 5/9] ref-filter: use generation number for --contains

2018-04-18 Thread Jakub Narebski
Here I can offer only the cursory examination, as I don't know this area of code in question. Derrick Stolee writes: > A commit A can reach a commit B only if the generation number of A > is larger than the generation number of B. This condition allows > significantly short-circuiting commit-gra

Re: [PATCH v3 6/9] commit: use generation numbers for in_merge_bases()

2018-04-18 Thread Jakub Narebski
Derrick Stolee writes: > The containment algorithm for 'git branch --contains' is different > from that for 'git tag --contains' in that it uses is_descendant_of() > instead of contains_tag_algo(). The expensive portion of the branch > algorithm is computing merge bases. > > When a commit-graph f

Re: [PATCH v3 7/9] commit: add short-circuit to paint_down_to_common()

2018-04-18 Thread Jakub Narebski
Derrick Stolee writes: > When running 'git branch --contains', the in_merge_bases_many() > method calls paint_down_to_common() to discover if a specific > commit is reachable from a set of branches. Commits with lower > generation number are not needed to correctly answer the > containment query

Re: [PATCH v3 8/9] commit-graph: always load commit-graph information

2018-04-18 Thread Jakub Narebski
Derrick Stolee writes: > Most code paths load commits using lookup_commit() and then > parse_commit(). In some cases, including some branch lookups, the commit > is parsed using parse_object_buffer() which side-steps parse_commit() in > favor of parse_commit_buffer(). > > With generation numbers

Re: [PATCH v3 0/9] Compute and consume generation numbers

2018-04-18 Thread Jakub Narebski
Derrick Stolee writes: > -- >8 -- > > This is the one of several "small" patches that follow the serialized > Git commit graph patch (ds/commit-graph) and lazy-loading trees > (ds/lazy-load-trees). > > As described in Documentation/technical/commit-graph.txt, the generation > number of a commit i

Re: [PATCH v3 7/9] commit: add short-circuit to paint_down_to_common()

2018-04-19 Thread Jakub Narebski
Derrick Stolee writes: > @@ -876,7 +886,7 @@ static struct commit_list *merge_bases_many(struct commit > *one, int n, struct co > return NULL; > } > > - list = paint_down_to_common(one, n, twos); > + list = paint_down_to_common(one, n, twos, 0); > >

Re: [RFC PATCH 02/12] commit-graph: add 'check' subcommand

2018-04-19 Thread Jakub Narebski
Derrick Stolee writes: > If the commit-graph file becomes corrupt, we need a way to verify > its contents match the object database. In the manner of 'git fsck' > we will implement a 'git commit-graph check' subcommand to report > all issues with the file. Bikeshed: should the subcommand be call

Re: [RFC PATCH 03/12] commit-graph: check file header information

2018-04-19 Thread Jakub Narebski
Derrick Stolee writes: > During a run of 'git commit-graph check', list the issues with the > header information in the commit-graph file. Some of this information > is inferred from the loaded 'struct commit_graph'. > > Signed-off-by: Derrick Stolee > --- > commit-graph.c | 29

Re: [RFC PATCH 04/12] commit-graph: parse commit from chosen graph

2018-04-19 Thread Jakub Narebski
Derrick Stolee writes: > Before checking a commit-graph file against the object database, we Actually there is quite a few checks more that can be done without accessing the object database... I'll take a look at later commits why this one is that relatively early in the series. > need to parse

Re: [RFC PATCH 05/12] commit-graph: check fanout and lookup table

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > While running 'git commit-graph check', verify that the object IDs > are listed in lexicographic order and that the fanout table correctly > navigates into that list of object IDs. All right. I think we can also sanity check the fanout table (for example that it has 256

Re: [PATCH v2 1/7] replace: "libify" create_graft()

2018-04-20 Thread Jakub Narebski
Junio C Hamano writes: > Johannes Schindelin writes: > >> It is quite convenient to simply die() in builtins, in the absence of >> proper exception handling, because it allows us to just go belly up >> without having to implement error handling chains. >> >> Of course, for reusable library functi

Re: [RFC PATCH 06/12] commit: force commit to parse from object database

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > In anticipation of checking commit-graph file contents against the > object database, create parse_commit_internal() to allow side-stepping > the commit-graph file and parse directly from the object database. Nitpick/Bikeshed painting: do we have any naming convention fo

Re: [RFC PATCH 07/12] commit-graph: load a root tree from specific graph

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > When lazy-loading a tree for a commit, it will be important to select > the tree from a specific struct commit_graph. Create a new method that > specifies the commit-graph file and use that in > get_commit_tree_in_graph(). > > Signed-off-by: Derrick Stolee Looks good to

Re: [RFC PATCH 08/12] commit-graph: verify commit contents against odb

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: One more check that could been done, and which do not require accessing the object database, would be testing correctness of the Large Edge List (EDGE) chunk. For each commit in the commit-graph (in the Commit Data (CDAT) chunk), check if it has more than two parents (if t

Re: [RFC PATCH 09/12] fsck: check commit-graph

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > If a commit-graph file exists, check its contents during 'git fsck'. Is it "if a commit-graph file exists", or is it core.commitGraph feature is turned on? > > Signed-off-by: Derrick Stolee > --- > builtin/fsck.c | 13 + > 1 file changed, 13 insertions(+)

Re: [RFC PATCH 10/12] commit-graph: add '--reachable' option

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > When writing commit-graph files, it can be convenient to ask for all > reachable commits (starting at the ref set) in the resulting file. This > is particularly helpful when writing to stdin is complicated, such as a > future integration with 'git gc' which will call > 'g

Re: [RFC PATCH 11/12] gc: automatically write commit-graph files

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file is a very helpful feature for speeding up git > operations. In order to make it more useful, write the commit-graph file > by default during standard garbage collection operations. > > Add a 'gc.commitGraph' config setting that triggers writing a > c

Re: [RFC PATCH 12/12] commit-graph: update design document

2018-04-20 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph feature is now integrated with 'fsck' and 'gc', > so remove those items from the "Future Work" section of the > commit-graph design document. See comments below, but this looks good to me. What is missing from the "Future Work" section (and which was so

Re: [PATCH 0/6] Compute and consume generation numbers

2018-04-21 Thread Jakub Narebski
Jakub Narebski writes: > Derrick Stolee writes: >> On 4/11/2018 3:32 PM, Jakub Narebski wrote: > >>> What would you suggest as a good test that could imply performance? The >>> Google Colab notebook linked to above includes a function to count >>> num

Re: [PATCH v3 7/9] commit: add short-circuit to paint_down_to_common()

2018-04-23 Thread Jakub Narebski
Derrick Stolee writes: > On 4/18/2018 7:19 PM, Jakub Narebski wrote: >> Derrick Stolee writes: >> > [...] >>> [...], and this saves time during 'git branch --contains' queries >>> that would otherwise walk "around" the commit we are in

Re: [PATCH v3 5/9] ref-filter: use generation number for --contains

2018-04-24 Thread Jakub Narebski
Derrick Stolee writes: > On 4/18/2018 5:02 PM, Jakub Narebski wrote: >> Derrick Stolee writes: >> >>> A commit A can reach a commit B only if the generation number of A >>> is larger than the generation number of B. This condition allows >>> signifi

Re: [PATCH v4 04/10] commit: use generations in paint_down_to_common()

2018-04-26 Thread Jakub Narebski
Junio C Hamano writes: > Derrick Stolee writes: > >> Define compare_commits_by_gen_then_commit_date(), which uses generation >> numbers as a primary comparison and commit date to break ties (or as a >> comparison when both commits do not have computed generation numbers). >> >> Since the commit-g

Re: [PATCH v4 04/10] commit: use generations in paint_down_to_common()

2018-04-28 Thread Jakub Narebski
Jakub Narebski writes: > Junio C Hamano writes: >> Derrick Stolee writes: [...] >>> +int compare_commits_by_gen_then_commit_date(const void *a_, const void >>> *b_, void *unused) >>> +{ >>> + const struct commit *a = a_, *b = b_; >>

Re: [PATCH v4 00/10] Compute and consume generation numbers

2018-04-28 Thread Jakub Narebski
Derrick Stolee writes: > As promised, here is the diff from v3. What is this strange string "   " in place of tabs in the interdiff? " " here is Unicode Character 'NO-BREAK SPACE' (U+00A0). Though it doesn't matter for viewing, my newsreader (Gnus from GNU Emacs) thinks that it is worth not

Re: [PATCH v4 01/10] ref-filter: fix outdated comment on in_commit_list

2018-04-28 Thread Jakub Narebski
Derrick Stolee writes: > The in_commit_list() method does not check the parents of > the candidate for containment in the list. Fix the comment > that incorrectly states that it does. > > Reported-by: Jakub Narebski > Signed-off-by: Derrick Stolee > --- > ref-filter.c

Re: [PATCH v4 02/10] commit: add generation number to struct commmit

2018-04-28 Thread Jakub Narebski
Derrick Stolee writes: > The generation number of a commit is defined recursively as follows: > > * If a commit A has no parents, then the generation number of A is one. > * If a commit A has parents, then the generation number of A is one > more than the maximum generation number among the par

Re: [PATCH v4 03/10] commit-graph: compute generation numbers

2018-04-29 Thread Jakub Narebski
Derrick Stolee writes: > While preparing commits to be written into a commit-graph file, compute > the generation numbers using a depth-first strategy. Sidenote: for generation numbers it does not matter if we use depth-first or breadth-first strategy, but it is more natural to use depth-first s

Re: [PATCH v4 04/10] commit: use generations in paint_down_to_common()

2018-04-29 Thread Jakub Narebski
Derrick Stolee writes: > Define compare_commits_by_gen_then_commit_date(), which uses generation > numbers as a primary comparison and commit date to break ties (or as a > comparison when both commits do not have computed generation numbers). All right, this looks reasonable thing to do when we

Re: [PATCH v4 05/10] commit-graph: always load commit-graph information

2018-04-29 Thread Jakub Narebski
Derrick Stolee writes: > Most code paths load commits using lookup_commit() and then > parse_commit(). And this automatically loads commit graph if needed, thanks to changes in parse_commit_gently(), which parse_commit() uses. > In some cases, including some branch lookups, the

Re: [PATCH v4 05/10] commit-graph: always load commit-graph information

2018-04-29 Thread Jakub Narebski
[Forgot about one thing] Derrick Stolee writes: > Create new load_commit_graph_info() method to fill in the information > for a commit that exists only in the commit-graph file. The above sentence is a bit hard to parse because of ambiguity: is it "the information" that exists only in the commi

Re: [PATCH v4 06/10] ref-filter: use generation number for --contains

2018-04-30 Thread Jakub Narebski
Derrick Stolee writes: > A commit A can reach a commit B only if the generation number of A > is strictly larger than the generation number of B. This condition > allows significantly short-circuiting commit-graph walks. > > Use generation number for '--contains' type queries. > > On a copy of th

Re: [PATCH v4 07/10] commit: use generation numbers for in_merge_bases()

2018-04-30 Thread Jakub Narebski
Derrick Stolee writes: > The containment algorithm for 'git branch --contains' is different > from that for 'git tag --contains' in that it uses is_descendant_of() > instead of contains_tag_algo(). The expensive portion of the branch > algorithm is computing merge bases. > > When a commit-graph f

Re: [PATCH v4 08/10] commit: add short-circuit to paint_down_to_common()

2018-04-30 Thread Jakub Narebski
Derrick Stolee writes: > When running 'git branch --contains', the in_merge_bases_many() > method calls paint_down_to_common() to discover if a specific > commit is reachable from a set of branches. Commits with lower > generation number are not needed to correctly answer the > containment query

Re: [PATCH v4 09/10] merge: check config before loading commits

2018-04-30 Thread Jakub Narebski
Derrick Stolee writes: > Now that we use generation numbers from the commit-graph, we must > ensure that all commits that exist in the commit-graph are loaded > from that file instead of from the object database. Since the > commit-graph file is only checked if core.commitGraph is true, we > must

Re: [PATCH v4 10/10] commit-graph.txt: update design document

2018-04-30 Thread Jakub Narebski
Derrick Stolee writes: > We now calculate generation numbers in the commit-graph file and use > them in paint_down_to_common(). > > Expand the section on generation numbers to discuss how the three > special generation numbers GENERATION_NUMBER_INFINITY, _ZERO, and > _MAX interact with other gene

Re: [PATCH v4 10/10] commit-graph.txt: update design document

2018-05-02 Thread Jakub Narebski
Derrick Stolee writes: > On 4/30/2018 7:32 PM, Jakub Narebski wrote: >> Derrick Stolee writes: [...] >>> - After computing and storing generation numbers, we must make graph >>> walks aware of generation numbers to gain the performance benefits they >>

Re: [PATCH v4 09/10] merge: check config before loading commits

2018-05-02 Thread Jakub Narebski
Derrick Stolee writes: > On 4/30/2018 6:54 PM, Jakub Narebski wrote: >> Derrick Stolee writes: >> >>> Now that we use generation numbers from the commit-graph, we must >>> ensure that all commits that exist in the commit-graph are loaded >>> from tha

Re: [PATCH v4 08/10] commit: add short-circuit to paint_down_to_common()

2018-05-02 Thread Jakub Narebski
Derrick Stolee writes: > On 4/30/2018 6:19 PM, Jakub Narebski wrote: >> Derrick Stolee writes: [...] >>> @@ -831,6 +834,13 @@ static struct commit_list *paint_down_to_common(struct >>> commit *one, int n, struc >>> struct commit_list

Re: [PATCH v4 03/10] commit-graph: compute generation numbers

2018-05-02 Thread Jakub Narebski
Derrick Stolee writes: > On 4/29/2018 5:08 AM, Jakub Narebski wrote: >> Derrick Stolee writes: [...] >> It is a bit strange to me that the code uses get_be32 for reading, but >> htonl for writing. Is Git tested on non little-endian machines, like >> big-endian

Re: [PATCH v5 00/11] Compute and consume generation numbers

2018-05-03 Thread Jakub Narebski
Derrick Stolee writes: > Most of the changes from v4 are cosmetic, but there is one new commit: > > commit: use generation number in remove_redundant() > > Other changes are non-functional, but do clarify things. I wonder if out perf framework in t/perf could help here to show performance

Re: [PATCH] gitweb: Measure offsets against UTF-8 flagged string

2018-05-03 Thread Jakub Narebski
Junio C Hamano writes: > Shin Kojima writes: > >> Offset positions should not be counted by byte length, but by actual >> character length. >> ... >> # escape tabs (convert tabs to spaces) >> sub untabify { >> -my $line = shift; >> +my $line = to_utf8(shift); >> >> while ((my $po

Re: [PATCH v5 09/11] commit: use generation number in remove_redundant()

2018-05-03 Thread Jakub Narebski
s spent walking the history to find the > set of merge bases before the remove_redundant() call. The > starting commits are closer together in the second example, > therefore more time is spent in remove_redundant(). The relative > change in performance differs as expected. > > Re

[RFC] Other chunks for commit-graph, part 1 - Bloom filters, topo order, etc.

2018-05-04 Thread Jakub Narebski
Hello, With early parts of commit-graph feature (ds/commit-graph and ds/lazy-load-trees) close to being merged into "master", see https://public-inbox.org/git/xmqq4ljtz87g@gitster-ct.c.googlers.com/ I think it would be good idea to think what other data could be added there to make Git even fa

Re: [RFC] Other chunks for commit-graph, part 1 - Bloom filters, topo order, etc.

2018-05-05 Thread Jakub Narebski
Ævar Arnfjörð Bjarmason writes: > On Fri, May 04 2018, Jakub Narebski wrote: > > (Just off-the cuff here and I'm surely about to be corrected by > Derrick...) > >> * What to do about merge commits, and octopus merges in particular? >> Should Bloom filter be stor

[RFC] Other chunks for commit-graph, part 2 - reachability indexes

2018-05-06 Thread Jakub Narebski
Hello, In previous email I wrote: JN> As this post is getting long, I'll post other ideas, about commit JN> labeling for faster reachability queries in a separate email. This is this email. Here is second part of series dedicated to discussing what other data, like various reachability indexes

Re: [RFC] Other chunks for commit-graph, part 1 - Bloom filters, topo order, etc.

2018-05-12 Thread Jakub Narebski
Derrick Stolee writes: > On 5/4/2018 3:40 PM, Jakub Narebski wrote: >> >> With early parts of commit-graph feature (ds/commit-graph and >> ds/lazy-load-trees) close to being merged into "master", see >> https://public-inbox.org/git/xmqq4ljtz87g@gitster-c

Re: [PATCH 00/12] Die commit->util, die!

2018-05-12 Thread Jakub Narebski
Nguyễn Thái Ngọc Duy writes: > There's not much to write here. It's basically a copy from 12/12: > > This 'util' pointer can be used for many different purposes, > controlled in different ways. Some are not even contained in a command > code, but buried deep in common code with no clue who will u

Re: [RFC] Other chunks for commit-graph, part 1 - Bloom filters, topo order, etc.

2018-05-14 Thread Jakub Narebski
Derrick Stolee writes: > On 5/12/2018 10:00 AM, Jakub Narebski wrote: >> Derrick Stolee writes: >>> On 5/4/2018 3:40 PM, Jakub Narebski wrote: >>>> >>>> With early parts of commit-graph feature (ds/commit-graph and >>>> ds/lazy-load-tree

Re: [PATCH v2 01/12] commit-graph: add 'verify' subcommand

2018-05-20 Thread Jakub Narebski
Derrick Stolee writes: > If the commit-graph file becomes corrupt, we need a way to verify > that its contents match the object database. In the manner of > 'git fsck' we will implement a 'git commit-graph verify' subcommand > to report all issues with the file. > > Add the 'verify' subcommand to

Re: [PATCH v2 02/12] commit-graph: verify file header information

2018-05-20 Thread Jakub Narebski
Derrick Stolee writes: > During a run of 'git commit-graph verify', list the issues with the > header information in the commit-graph file. Some of this information > is inferred from the loaded 'struct commit_graph'. Some header > information is checked as part of load_commit_graph_one(). > > Si

Re: [PATCH v2 03/12] commit-graph: test that 'verify' finds corruption

2018-05-21 Thread Jakub Narebski
Derrick Stolee writes: > Add test cases to t5318-commit-graph.sh that corrupt the commit-graph > file and check that the 'git commit-graph verify' command fails. These > tests verify the header and chunk information is checked carefully. > > Helped-by: Martin Ågren > Signed-off-by: Derrick Stole

Re: commit-graph: change in "best" merge-base when ambiguous

2018-05-24 Thread Jakub Narebski
Derrick Stolee writes: > On 5/22/2018 1:39 AM, Michael Haggerty wrote: >> On 05/21/2018 08:10 PM, Derrick Stolee wrote: >>> [...] >>> In the Discussion section of the `git merge-base` docs [1], we have the >>> following: >>> >>>     When the history involves criss-cross merges, there can be more

Re: RFC: New reference iteration paradigm

2018-05-26 Thread Jakub Narebski
Jeff King writes: > On Thu, Mar 31, 2016 at 11:01:44AM -0700, Junio C Hamano wrote: >> Michael Haggerty writes: >> >>> the backend now has to implement >>> struct ref_iterator *ref_iterator_begin_fn(const char *submodule, const char *prefix, >

Re: [PATCH v3 02/20] commit-graph: fix GRAPH_MIN_SIZE

2018-05-26 Thread Jakub Narebski
Derrick Stolee writes: > The GRAPH_MIN_SIZE macro should be the smallest size of a parsable > commit-graph file. However, the minimum number of chunks was wrong. > It is possible to write a commit-graph file with zero commits, and > that violates this macro's value. > > Rewrite the macro, and use

Re: [PATCH v3 03/20] commit-graph: parse commit from chosen graph

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > Before verifying a commit-graph file against the object database, we > need to parse all commits from the given commit-graph file. Create > parse_commit_in_graph_one() to target a given struct commit_graph. If I understand it properly the problem is that when verifying a

Re: [PATCH v3 04/20] commit: force commit to parse from object database

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > In anticipation of verifying commit-graph file contents against the > object database, create parse_commit_internal() to allow side-stepping > the commit-graph file and parse directly from the object database. > > Due to the use of generation numbers, this method should n

Re: [PATCH v3 05/20] commit-graph: load a root tree from specific graph

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > When lazy-loading a tree for a commit, it will be important to select > the tree from a specific struct commit_graph. Create a new method that > specifies the commit-graph file and use that in > get_commit_tree_in_graph(). Is this for the same reason why parse_commit_in_

Re: [PATCH v3 06/20] commit-graph: add 'verify' subcommand

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > If the commit-graph file becomes corrupt, we need a way to verify > that its contents match the object database. In the manner of > 'git fsck' we will implement a 'git commit-graph verify' subcommand > to report all issues with the file. > > Add the 'verify' subcommand to

Re: [PATCH v3 07/20] commit-graph: verify catches corrupt signature

2018-05-28 Thread Jakub Narebski
Derrick Stolee writes: > This is the first of several commits that add a test to check that > 'git commit-graph verify' catches corruption in the commit-graph > file. The first test checks that the command catches an error in > the file signature. This is a check that exists in the existing > com

Re: [PATCH v3 08/20] commit-graph: verify required chunks are present

2018-05-28 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file requires the following three chunks: > > * OID Fanout > * OID Lookup > * Commit Data > > If any of these are missing, then the 'verify' subcommand should > report a failure. This includes the chunk IDs malformed or the > chunk count is truncated. Mi

Re: [PATCH v3 09/20] commit-graph: verify corrupt OID fanout and lookup

2018-05-30 Thread Jakub Narebski
Derrick Stolee writes: > In the commit-graph file, the OID fanout chunk provides an index into > the OID lookup. The 'verify' subcommand should find incorrect values > in the fanout. > > Similarly, the 'verify' subcommand should find out-of-order values in > the OID lookup. O.K., the OID Lookup

Re: [PATCH v3 10/20] commit-graph: verify objects exist

2018-05-30 Thread Jakub Narebski
Derrick Stolee writes: > In the 'verify' subcommand, load commits directly from the object > database to ensure they exist. Parse by skipping the commit-graph. All right, before we check that the commit data matches, we need to check that all the commits in cache (in the serialized commit graph)

Re: [PATCH v3 11/20] commit-graph: verify root tree OIDs

2018-05-30 Thread Jakub Narebski
Derrick Stolee writes: > The 'verify' subcommand must compare the commit content parsed from the > commit-graph and compare it against the content in the object database. You have "compare" twice in the above sentence. > Use lookup_commit() and parse_commit_in_graph_one() to parse the commits >

Re: [PATCH v3 12/20] commit-graph: verify parent list

2018-06-01 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file stores parents in a two-column portion of the > commit data chunk. If there is only one parent, then the second column > stores 0x to indicate no second parent. All right, it is certainly nice to have this information, but isn't it something

Re: How to resume broke clone ?

2013-11-28 Thread Jakub Narebski
zhifeng hu ancientrocklab.com> writes: > > Once using git clone —depth or git fetch —depth, > While you want to move backward. > you may face problem > > git fetch --depth=105 > error: Could not read 483bbf41ca5beb7e38b3b01f21149c56a1154b7a > error: Could not read aacb82de3ff8ae7b0a9e4cfec16c1

Re: [PATCH v2 09/10] builtin/replace: unset read_replace_refs

2013-12-07 Thread Jakub Narebski
Christian Couder tuxfamily.org> writes: > > When checking to see if some objects are of the same type > and when displaying the type of objects, git replace uses > the sha1_object_info() function. > > Unfortunately this function by default respects replace > refs, so instead of the type of a re

Re: [PATCH] RelNotes/1.8.5: direct script writers to "git status --porcelain"

2013-09-26 Thread Jakub Narebski
t;git status --porcelain", which is both easier to parse and stable, > + instead. Good addition. Perhaps "to use instead ..." would be easier to understand than proposed "to use ..., instead." (with "..." being one line long). -- Jakub Narebski. -- T

Re: Setting per-repository configuration for git

2013-10-25 Thread Jakub Narebski
Jeff King peff.net> writes: > On Fri, Oct 25, 2013 at 09:12:35AM +0200, Jeremy Rosen wrote: > > > however I can't find a way to have the repository's configuration > > saved and transmited with the repository in a way similar to how > > .gitignore is transmitted... > > [...] > Two, the config i

Re: Feature request:

2013-07-03 Thread Jakub Narebski
[I'm sorry about breaking Cc: chain - responding via GMane web interface] Junio C Hamano pobox.com> writes: > Ed Hutchins demeterr.com> writes: > > > I'm not trying to change the way git does things (which works perfectly > > well), I'm asking for some extra information to be added to the commi

Re: [PATCH v3 0/6]

2013-07-23 Thread Jakub Narebski
Junio C Hamano pobox.com> writes: > > This is mostly unchanged since the previous round, except that > > * The option is spelled "--force-with-lease=:". >Nobody liked "cas" as it was too technical, many disliked >"lockref" because "lock" sounded as if push by others were >excluded

Re: How to hierarchically merge from the root to the leaf of a branch tree? (Patch stack management)

2013-08-01 Thread Jakub Narebski
Jens Müller tessarakt.de> writes: > > Hi all! > > I mainly use Git for version control, but have also tried out Mercurial. > While I don't really like Mercurial in general, the idea of maintaining > clearly separated patches with Mercurial Queues (MQ) is quite appealing. > Therefore, I am looki

Re: Has anyone tried to implement git grep --blame?

2013-05-30 Thread Jakub Narebski
Ævar Arnfjörð Bjarmason gmail.com> writes: > This would be so much more convenient if git-grep supported it natively: > > $ git grep -n 'if \(0\)' | perl -pe's/([^:]+):([^:]+).*/`git blame -L > $2,$2 $1`/se' > d18f76dc (Ævar Arnfjörð Bjarmason 2010-08-17 09:24:38 + 2278) if (0) > 65648283

Re: Git exile Issues

2013-06-08 Thread Jakub Narebski
Sudhir Kumar gmail.com> writes: > > Hey Git Experts, > > I need your advice. I have lot of png/jpg images in my codebase (which > is currently under git) which causes the repo size to be very heavy. > We have migrated these images to a storage server using git exile > technique. This has been w

Re: What's cooking in git.git (Jun 2013, #04; Tue, 11)

2013-06-12 Thread Jakub Narebski
Junio C Hamano pobox.com> writes: > * rr/remove-contrib-some (2013-06-02) 1 commit > (merged to 'next' on 2013-06-05 at fc15705) > + contrib: remove continuous/ and patches/ > > Remove stale contrib/ material. > > Will merge to 'master'. What about contrib/blameview by Aneesh Kumar K.V a

Re: [PATCH] instaweb: make the perl path configurable

2013-06-12 Thread Jakub Narebski
Charles McGarvey brokenzipper.com> writes: > It is convenient for the user to be able to customize the path to perl if they > do not want to use the system perl. This may be the case, for example, if the > user wants to use the plackup httpd but its extra dependencies are not > installed in the

Re: Exact format of tree objets

2013-06-12 Thread Jakub Narebski
Junio C Hamano pobox.com> writes: > Chico Sokol gmail.com> writes: > > > Is there any official documentation of tree objets format? Are tree > > objects encoded specially in some way? How can I parse the inflated > > contents of a tree object? > > > > We're suspecting that there is some kind of

Re: [PATCH] Documentation/CommunityGuidelines

2013-06-12 Thread Jakub Narebski
Philip Oakley iee.org> writes: > From: "Michael Haggerty" alum.mit.edu> > Sent: Tuesday, June 11, 2013 7:52 PM > >> As my mother would say, "politeness costs nothing" > > > > Does your mother program C? We could use her around here > > I think she programmed in Smalltalk and CleanYourRoom.

Re: [PATCH] gitweb: URL-decode $my_url/$my_uri when stripping PATH_INFO

2012-08-15 Thread Jakub Narebski
akage like this is something we can catch in one of > the t95xx series of tests, though. No, it is unfortunately not possible with current test infrastructure for gitweb. The gitweb_run from t/gitweb-lib.sh allows to set PATH_INFO and QUERY_STRING, but does not allow to set up URL. That m

<    1   2