Hello, My previous testing was done entirely based on the nova repo and jgit (without gerrit) so I believe there was still some concern whether we should allow gerrit to do the gc since that might do something bad to our repos. So going a step further with testing, I have setup the provided snapshot of the nova repo[1] on review-dev.o.o[2] and basically re-ran the my tests with `gerrit gc`
Additional info: * I ran a manual `gerrit gc` on the 'nova-gc' repo. From the source code[3] it looks like Gerrit is just running a `jgit gc` on it. I have confirmed that the repo seems to work perfectly fine after running it thru a gerrit gc. I tested clone, push, fetch, and various things thru the UI (reviews, comments, abandon, etc...) on the repro after the gc. * I cloned locally on review-dev.o.o to test the performance and the result is pretty much the same as my previous testing. Cloned 4 times with each repo: nova-no-gc: 3m44.551s, 2m55.797s, 2m51.078s, 2m57.749s nova-gc: 0m28.824s, 0m28.960s, 0m29.359s, 0m31.943s * I also tested fetch and push again and result were similar to previous test as well. These operations were pretty quick when run locally but in general both were faster with 'nova-gc' repo. * Looking at the repo files, the objects were significantly pruned which saves disk space. I actually expected the refs to be completely cleaned up however `gerrit gc` doesn't actually clean out all of the refs but only reduces it. It does create the packed-refs file which is probably the thing that improves the performance of the clone operation. before gerrit gc: gerrit2@review-dev:~$ du -hsx review_site/git/kdtest/nova-gc.git/* | sort -rh |head -10 6.4G review_site/git/kdtest/nova-no-gc.git/objects 468M review_site/git/kdtest/nova-no-gc.git/refs 6.2M review_site/git/kdtest/nova-no-gc.git/info 2.2M review_site/git/kdtest/nova-no-gc.git/logs 4.0K review_site/git/kdtest/nova-no-gc.git/hooks 4.0K review_site/git/kdtest/nova-no-gc.git/HEAD 4.0K review_site/git/kdtest/nova-no-gc.git/description 4.0K review_site/git/kdtest/nova-no-gc.git/config 4.0K review_site/git/kdtest/nova-no-gc.git/branches after gerrit gc: gerrit2@review-dev:~$ du -hsx review_site/git/kdtest/nova-gc.git/* | sort -rh |head -10 475M review_site/git/kdtest/nova-gc.git/objects 86M review_site/git/kdtest/nova-gc.git/refs 6.2M review_site/git/kdtest/nova-gc.git/packed-refs 6.1M review_site/git/kdtest/nova-gc.git/info 2.2M review_site/git/kdtest/nova-gc.git/logs 4.0K review_site/git/kdtest/nova-gc.git/hooks 4.0K review_site/git/kdtest/nova-gc.git/HEAD 4.0K review_site/git/kdtest/nova-gc.git/config 4.0K review_site/git/kdtest/nova-gc.git/branches 0 review_site/git/kdtest/nova-gc.git/description [1] http://tarballs.openstack.org/ci/nova.git.tar.bz2 [2] https://review-dev.openstack.org/#/admin/projects/?filter=kdtest [3] http://git.openstack.org/cgit/openstack-infra/gerrit/tree/gerrit-server/src/main/java/com/google/gerrit/server/git/GarbageCollection.java?h=openstack/2.11.4#n83 On Fri, Mar 25, 2016 at 5:47 PM, Zaro <zaro0...@gmail.com> wrote: > So I've been researching this and I've found that there is a > significant performance improvement after running git gc on this nova > repro. Below are my results. > > File sizes of repo as-is: > ~/nova.git.orig$ du -hsx * | sort -r | head -10 > 6.4G objects > 6.1M info > 4.0K config > 4.0K HEAD > 382M refs > 2.1M logs > 0B hooks > 0B description > 0B branches > > Note that the repro as-is has already been thru a 'git repack -afd'. > > > File sizes after running 'jgit gc': > ~/nova.git.test$ du -hsx * | sort -r | head -10 > 6.1M packed-refs > 6.1M info > 420M objects > 4.0K config > 4.0K HEAD > 2.1M logs > 0B refs > 0B hooks > 0B description > 0B branches > > The result is that the gc cleans up the objects (6.4G -> 420M) and > moves the loose ref objects from 'refs' dir to a 'packed-refs' file > (382M -> 6.1M). > > Note that I'm using jgit because that's what Gerrit would use to do > the 'gc'. The jgit version is 4.0.1.201506240215-r which is the one > that's packaged with our current version of Gerrit > (2.11.4-11-ga14450f) on review.o.o > > > Here I've tested the performance of the git clone, fetch and push > before and after running 'jgit gc': > > `git clone` > ------------ > before: > real 3m30.163s > user 0m2.020s > sys 3m15.087s > > after: > real 0m0.925s > user 0m0.406s > sys 0m0.621s > > > `git fetch origin stable/liberty` > --------------------------------- > before: > real 0m4.271s > user 0m0.701s > sys 0m2.949s > > after: > real 0m0.686s > user 0m0.348s > sys 0m0.307s > > > `git push origin HEAD:refs/for/master` > -------------------------------------- > before: > real 0m36.454s > user 0m5.346s > sys 0m27.598s > > after: > real 0m16.588s > user 0m11.731s > sys 0m3.218s > > Note: I pushed the exact same change for both scenarios. > > > Conclusion: > The results indicate that it would be very advantages to run 'git gc' > for both file size reduction and improved performance. Below are > additional resources that I've found on the internet that seems to > back up my results. > > > > references: > > This says that one-file-per-ref format both wastes storage and hurts > performance: https://git-scm.com/docs/git-pack-refs > > This outlines some of the benefits and drawbacks of packed-refs file: > https://www.mail-archive.com/git%40vger.kernel.org/msg65722.html > > Info on speeding up clones/fetches with pack bitmaps: > https://www.mail-archive.com/git%40vger.kernel.org/msg65571.html > > On Fri, Jan 8, 2016 at 12:13 PM, James E. Blair <cor...@inaugust.com> wrote: >> Hi, >> >> With the new version of Gerrit offering built-in "git gc" capability, we >> looked at the current state of our git repo maintenance. We run "git >> repack -afd" weekly in an attempt to produce the smallest packfiles >> possible, but it does not prune loose objects, which seems to be the >> main thing "git gc" does that we are missing. >> >> Some (relatively) quick experimentation suggests that various >> combinations of "git gc", "git repack", "git prune", "git prune-packed" >> all have effects on the overall repo size, the number of pack files, and >> the number of loose objects. >> >> However, we don't just want to find the thing that makes the smallest >> repo size (that's easy: "git prune; git gc" -- 394M for nova; one >> packfile with all objects and one packed-refs file with all refs) >> because this repo is used as the basis of all of our mirrors and is >> accessed over several protocols. It's not immediately clear what the >> right optimization is for our situation -- we don't necessarily want to >> trade on-disk size for reduced network performance. Even the packing of >> refs isn't entirely straightforward -- while we haven't needed to for >> some time, we have, in the past removed refs. >> >> We're looking for a volunteer to really dig into this problem and >> thoroughly evaluate the implications of different ways of optimizing the >> repo. If you're interested, you can download a snapshot of the full >> nova repository from Gerrit (it is a point-in-time snapshot and will not >> be updated) at this URL: >> >> http://tarballs.openstack.org/ci/nova.git.tar.bz2 >> >> Please follow up this message if you are interested and with any >> findings. >> >> Thanks, >> >> Jim >> >> _______________________________________________ >> OpenStack-Infra mailing list >> OpenStack-Infra@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra _______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
