Re: [OpenStack-Infra] gate tests for Rally failing
+ Andrey Andrey can you help? Best regards, Boris Pavlovic On Thu, Oct 20, 2016 at 12:01 PM, Jayathirth, Pramod Raghavendra < pramod.raghavendra.jayathi...@intel.com> wrote: > Hi team, > > > > I am Pramod and I am a developer for the networking-odl project. The gate > tests for Rally has been failing for all the commits from a long time in > our project. We would like to make those rally tests as voting from now. > Can I get the team’s help in resolving the issue?. I am having trouble > where to start with since I am not sure if it is the problem with the tests > or the infra. > > > > The link for the failing jobs is https://review.openstack.org/#/c/364040/9 > . > > The jobs which I would like to fix are :- gate-rally-dsvm-networking- > odl-beryllium-snapshot-nv > <http://logs.openstack.org/40/364040/9/check/gate-rally-dsvm-networking-odl-beryllium-snapshot-nv/c4f49ad/>, > gate-rally-dsvm-networking-odl-boron-snapshot-nv > <http://logs.openstack.org/40/364040/9/check/gate-rally-dsvm-networking-odl-boron-snapshot-nv/b9d41e7/>, > gate-rally-dsvm-networking-odl-carbon-snapshot-nv > <http://logs.openstack.org/40/364040/9/check/gate-rally-dsvm-networking-odl-carbon-snapshot-nv/093a44e/>. > > > Thanks and Regards, > > Pramod Raghavendra Jayathirth > > > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Change Python 2.6 jobs to non-voting
Hi Christian, I don't want to drop py26 support in Rally, Loggas and OSprofiler. So please don't change these projects. Best regards, Boris Pavlovic On Thu, Oct 9, 2014 at 11:54 AM, Christian Berendt wrote: > Because the Python 2.6 support should be dropped I think it makes sense > to change all existing Python 2.6 jobs to non-voting. > > Christian. > > -- > Christian Berendt > Cloud Computing Solution Architect > Mail: bere...@b1-systems.de > > B1 Systems GmbH > Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de > GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537 > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Cross-Repo Dependencies in Zuul
James,
Awesome! Amazing! You guys rock!=)
Best regards,
Boris Pavlovic
On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair wrote:
> Hi,
>
> We have added support for cross-repo dependencies (CRD) in Zuul. The
> important bits:
>
> * To use them, include "Depends-On: " in the footer of
> your commit message. Use the full Change-ID ('I' + 40 characters).
>
> * These are one-way dependencies only -- do not create a cycle.
>
> * This is what all the grey dots and lines are in the check pipeline.
>
> Cross-Repo Dependencies Explained
> =
>
> There are two behaviors that might go by the name "cross-repo
> dependencies". We call them one-way and multi-way.
>
> Multi-way CRD allow for bidirectional links between changes. For
> instance, A depends on B and B depends on A. The theory there is that
> both would be tested together and merged as a unit. This is _not_ what
> we have implemented in Zuul. Discussions over the past two years have
> revealed that this type of behavior could cause problems for continuous
> deploments as it means that two components must be upgraded
> simultaneously. Not supporting this behavior is a choice we have made.
>
> One-way CRD behaves like a directed acyclic graph (DAG), like git
> itself, to indicate a one-way dependency relationship between changes in
> different git repos. Change A may depend on B, but B may not depend on
> A. This is what we have implemented in Zuul.
>
> Gate Pipeline
> =
>
> When Zuul sees CRD changes, it serializes them in the usual manner when
> enqueuing them into a pipeline. This means that if change A depends on
> B, then when they are added to the gate pipeline, B will appear first
> and A will follow. If tests for B fail, both B and A will be removed
> from the pipeline, and it will not be possible for A to merge until B
> does.
>
> Note that if changes with CRD do not share a change queue (such as the
> "integrated gate" then Zuul is unable to enqueue them together, and the
> first will be required to merge before the second is enqueued.
>
> Check Pipeline
> ==
>
> When changes are enqueued into the check pipeline, all of the related
> dependencies (both normal git-dependencies that come from parent commits
> as well as CRD changes) appear in a dependency graph, as in gate. This
> means that even in the check pipeline, your change will be tested with
> its dependency. So changes that were previously unable to be fully
> tested until a related change landed in a different repo may now be
> tested together from the start.
>
> All of the changes are still independent (so you will note that the
> whole pipeline does not share a graph as in gate), but for each change
> tested, all of its dependencies are visually connected to it, and they
> are used to construct the git references that Zuul uses when testing.
> When looking at this graph on the status page, you will note that the
> dependencies show up as grey dots, while the actual change tested shows
> up as red or green. This is to indicate that the grey changes are only
> there to establish dependencies. Even if one of the dependencies is
> also being tested, it will show up as a grey dot when used as a
> dependency, but separately and additionally will appear as its own red
> or green dot for its test.
>
> (If you don't see grey dots on the status page, reload the page to get
> the latest version.)
>
> Multiple Changes
>
>
> A Gerrit change ID may refer to multiple changes (on multiple branches
> of the same project, or even multiple projects). In these cases, Zuul
> will treat all of the changes with that change ID as dependencies. So
> if you say that a tempest change Depends-On a change ID that has changes
> in nova master and nova stable/juno, then when testing the tempest
> change, both nova changes will be applied, and when deciding whether the
> tempest change can merge, both changes must merge ahead of it.
>
> A change may depend on more than one Gerrit change ID as well. So it is
> possible for a change in tempest to depend on a change in devstack and a
> change in nova. Simply add more "Depends-On:" lines to the footer.
>
> Cycles
> ==
>
> If a cycle is created by use of CRD, Zuul will abort its work very
> early. There will be no message in Gerrit and no changes that are part
> of the cycle will be enqueued into any pipeline. This is to protect
> Zuul from infinite loops. I hope that we can improve this to at least
> leave a message in Gerrit in the future. But in the meantime, please be
> cognizant of this and do not create dependency cycles with Depe
Re: [OpenStack-Infra] Cross-Repo Dependencies in Zuul
Anita,
By "guys" I refer to whole infra team including women...
P.S. I thought that "guys" is gender-neutral....
Best regards,
Boris Pavlovic
On Wed, Feb 11, 2015 at 1:34 AM, Anita Kuno wrote:
> On 02/10/2015 05:26 PM, Boris Pavlovic wrote:
> > James,
> >
> > Awesome! Amazing! You guys rock!=)
> Hopefully the women do too.
>
> Anita.
> >
> > Best regards,
> > Boris Pavlovic
> >
> >
> >
> > On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair
> wrote:
> >
> >> Hi,
> >>
> >> We have added support for cross-repo dependencies (CRD) in Zuul. The
> >> important bits:
> >>
> >> * To use them, include "Depends-On: " in the footer of
> >> your commit message. Use the full Change-ID ('I' + 40 characters).
> >>
> >> * These are one-way dependencies only -- do not create a cycle.
> >>
> >> * This is what all the grey dots and lines are in the check pipeline.
> >>
> >> Cross-Repo Dependencies Explained
> >> =
> >>
> >> There are two behaviors that might go by the name "cross-repo
> >> dependencies". We call them one-way and multi-way.
> >>
> >> Multi-way CRD allow for bidirectional links between changes. For
> >> instance, A depends on B and B depends on A. The theory there is that
> >> both would be tested together and merged as a unit. This is _not_ what
> >> we have implemented in Zuul. Discussions over the past two years have
> >> revealed that this type of behavior could cause problems for continuous
> >> deploments as it means that two components must be upgraded
> >> simultaneously. Not supporting this behavior is a choice we have made.
> >>
> >> One-way CRD behaves like a directed acyclic graph (DAG), like git
> >> itself, to indicate a one-way dependency relationship between changes in
> >> different git repos. Change A may depend on B, but B may not depend on
> >> A. This is what we have implemented in Zuul.
> >>
> >> Gate Pipeline
> >> =
> >>
> >> When Zuul sees CRD changes, it serializes them in the usual manner when
> >> enqueuing them into a pipeline. This means that if change A depends on
> >> B, then when they are added to the gate pipeline, B will appear first
> >> and A will follow. If tests for B fail, both B and A will be removed
> >> from the pipeline, and it will not be possible for A to merge until B
> >> does.
> >>
> >> Note that if changes with CRD do not share a change queue (such as the
> >> "integrated gate" then Zuul is unable to enqueue them together, and the
> >> first will be required to merge before the second is enqueued.
> >>
> >> Check Pipeline
> >> ==
> >>
> >> When changes are enqueued into the check pipeline, all of the related
> >> dependencies (both normal git-dependencies that come from parent commits
> >> as well as CRD changes) appear in a dependency graph, as in gate. This
> >> means that even in the check pipeline, your change will be tested with
> >> its dependency. So changes that were previously unable to be fully
> >> tested until a related change landed in a different repo may now be
> >> tested together from the start.
> >>
> >> All of the changes are still independent (so you will note that the
> >> whole pipeline does not share a graph as in gate), but for each change
> >> tested, all of its dependencies are visually connected to it, and they
> >> are used to construct the git references that Zuul uses when testing.
> >> When looking at this graph on the status page, you will note that the
> >> dependencies show up as grey dots, while the actual change tested shows
> >> up as red or green. This is to indicate that the grey changes are only
> >> there to establish dependencies. Even if one of the dependencies is
> >> also being tested, it will show up as a grey dot when used as a
> >> dependency, but separately and additionally will appear as its own red
> >> or green dot for its test.
> >>
> >> (If you don't see grey dots on the status page, reload the page to get
> >> the latest version.)
> >>
> >> Multiple Changes
> >>
> >>
> >> A Gerrit change ID may refer to multiple changes (on multiple branches
> >> of the same project, or even multiple projects). In
Re: [OpenStack-Infra] Cross-Repo Dependencies in Zuul
Anita,
Sorry for my bad english and not proper usage of "guys".
But the only thing that I was trying to say is that infra team rocks.
(that's all)
Best regards,
Boris Pavlovic
On Wed, Feb 11, 2015 at 1:44 AM, Anita Kuno wrote:
> On 02/10/2015 05:42 PM, Boris Pavlovic wrote:
> > Anita,
> >
> >
> > By "guys" I refer to whole infra team including women...
> > P.S. I thought that "guys" is gender-neutral
> It isn't.
>
> Folks, group, team are all gender neutral.
>
> A recent tc meeting likes 'y'all' for group gender neutral terms.
>
> Thanks,
> Anita.
> >
> >
> > Best regards,
> > Boris Pavlovic
> >
> > On Wed, Feb 11, 2015 at 1:34 AM, Anita Kuno
> wrote:
> >
> >> On 02/10/2015 05:26 PM, Boris Pavlovic wrote:
> >>> James,
> >>>
> >>> Awesome! Amazing! You guys rock!=)
> >> Hopefully the women do too.
> >>
> >> Anita.
> >>>
> >>> Best regards,
> >>> Boris Pavlovic
> >>>
> >>>
> >>>
> >>> On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair
> >> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> We have added support for cross-repo dependencies (CRD) in Zuul. The
> >>>> important bits:
> >>>>
> >>>> * To use them, include "Depends-On: " in the footer
> of
> >>>> your commit message. Use the full Change-ID ('I' + 40 characters).
> >>>>
> >>>> * These are one-way dependencies only -- do not create a cycle.
> >>>>
> >>>> * This is what all the grey dots and lines are in the check pipeline.
> >>>>
> >>>> Cross-Repo Dependencies Explained
> >>>> =
> >>>>
> >>>> There are two behaviors that might go by the name "cross-repo
> >>>> dependencies". We call them one-way and multi-way.
> >>>>
> >>>> Multi-way CRD allow for bidirectional links between changes. For
> >>>> instance, A depends on B and B depends on A. The theory there is that
> >>>> both would be tested together and merged as a unit. This is _not_
> what
> >>>> we have implemented in Zuul. Discussions over the past two years have
> >>>> revealed that this type of behavior could cause problems for
> continuous
> >>>> deploments as it means that two components must be upgraded
> >>>> simultaneously. Not supporting this behavior is a choice we have
> made.
> >>>>
> >>>> One-way CRD behaves like a directed acyclic graph (DAG), like git
> >>>> itself, to indicate a one-way dependency relationship between changes
> in
> >>>> different git repos. Change A may depend on B, but B may not depend
> on
> >>>> A. This is what we have implemented in Zuul.
> >>>>
> >>>> Gate Pipeline
> >>>> =
> >>>>
> >>>> When Zuul sees CRD changes, it serializes them in the usual manner
> when
> >>>> enqueuing them into a pipeline. This means that if change A depends
> on
> >>>> B, then when they are added to the gate pipeline, B will appear first
> >>>> and A will follow. If tests for B fail, both B and A will be removed
> >>>> from the pipeline, and it will not be possible for A to merge until B
> >>>> does.
> >>>>
> >>>> Note that if changes with CRD do not share a change queue (such as the
> >>>> "integrated gate" then Zuul is unable to enqueue them together, and
> the
> >>>> first will be required to merge before the second is enqueued.
> >>>>
> >>>> Check Pipeline
> >>>> ==
> >>>>
> >>>> When changes are enqueued into the check pipeline, all of the related
> >>>> dependencies (both normal git-dependencies that come from parent
> commits
> >>>> as well as CRD changes) appear in a dependency graph, as in gate.
> This
> >>>> means that even in the check pipeline, your change will be tested with
> >>>> its dependency. So changes that were previously unable to be fully
> >>>> tested until a related change landed in a different repo may now be
> >>>> tested together from the start.
> >>>>
> &g
Re: [OpenStack-Infra] Gerrit downtime on Friday 2015-04-17 at 22:00 UTC
James, What about stackforge/rally -> openstack/rally ? Best regards, Boris Pavlovic On Wed, Apr 15, 2015 at 12:57 AM, James E. Blair wrote: > On Friday, April 17 at 22:00 UTC Gerrit will be unavailable for about 2 > hours while we rename some projects and perform some database > maintenance. > > Existing reviews, project watches, etc, should all be carried > over. Currently, we plan on renaming the following projects: > > stackforge/murano -> openstack/murano > stackforge/murano-agent-> openstack/murano-agent > stackforge/murano-apps -> openstack/murano-apps > stackforge/murano-dashboard-> openstack/murano-dashboard > stackforge/murano-deployment -> openstack/murano-deployment > stackforge/murano-specs-> openstack/murano-specs > stackforge/python-muranoclient -> openstack/python-muranoclient > stackforge/murano-docs -> stackforge-attic/murano-docs > > stackforge/congress -> openstack/congress > stackforge/congress-specs-> openstack/congress-specs > stackforge/python-congressclient -> openstack/python-congressclient > > stackforge/tuskar-ui-extras -> stackforge-attic/tuskar-ui-extras > > We also plan to transform some database fields in order to support > Unicode comments. > > If you have any questions about the maintenance, please reply here or > contact us in #openstack-infra on Freenode. > > -Jim > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Gerrit downtime on Friday 2015-04-17 at 22:00 UTC
Anita, Thank you. I added patch to that list. Best regards, Boris Pavlovic On Wed, Apr 15, 2015 at 1:02 AM, Anita Kuno wrote: > On 04/14/2015 05:59 PM, Boris Pavlovic wrote: > > James, > > > > What about stackforge/rally -> openstack/rally ? > > > > Best regards, > > Boris Pavlovic > Was the patch for the rename listed here? > > https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_Renames > > This is where we collect our rename patches to schedule them. > > If you have the patch ready and list it in the above slot in the > wikipage we can do the rename with the others in the same maintenance > window. > > Thanks, > Anita. > > > > On Wed, Apr 15, 2015 at 12:57 AM, James E. Blair > > wrote: > > > >> On Friday, April 17 at 22:00 UTC Gerrit will be unavailable for about 2 > >> hours while we rename some projects and perform some database > >> maintenance. > >> > >> Existing reviews, project watches, etc, should all be carried > >> over. Currently, we plan on renaming the following projects: > >> > >> stackforge/murano -> openstack/murano > >> stackforge/murano-agent-> openstack/murano-agent > >> stackforge/murano-apps -> openstack/murano-apps > >> stackforge/murano-dashboard-> openstack/murano-dashboard > >> stackforge/murano-deployment -> openstack/murano-deployment > >> stackforge/murano-specs-> openstack/murano-specs > >> stackforge/python-muranoclient -> openstack/python-muranoclient > >> stackforge/murano-docs -> stackforge-attic/murano-docs > >> > >> stackforge/congress -> openstack/congress > >> stackforge/congress-specs-> openstack/congress-specs > >> stackforge/python-congressclient -> openstack/python-congressclient > >> > >> stackforge/tuskar-ui-extras -> stackforge-attic/tuskar-ui-extras > >> > >> We also plan to transform some database fields in order to support > >> Unicode comments. > >> > >> If you have any questions about the maintenance, please reply here or > >> contact us in #openstack-infra on Freenode. > >> > >> -Jim > >> > >> ___ > >> OpenStack-Infra mailing list > >> OpenStack-Infra@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > >> > > > > > > > > ___ > > OpenStack-Infra mailing list > > OpenStack-Infra@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > > > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Gerrit downtime on Friday 2015-04-17 at 22:00 UTC
Anita, I'll do my best Best regards, Boris Pavlovic On Wed, Apr 15, 2015 at 1:11 AM, Anita Kuno wrote: > On 04/14/2015 06:06 PM, Boris Pavlovic wrote: > > Anita, > > > > Thank you. > > I added patch to that list. > > > > Best regards, > > Boris Pavlovic > Thank you, Boris. > > Just be responsive to reviews on the patch and it would help if someone > from the team was available during that time to answer any last minute > questions that may arise. > > Thanks, > Anita. > > > > On Wed, Apr 15, 2015 at 1:02 AM, Anita Kuno > wrote: > > > >> On 04/14/2015 05:59 PM, Boris Pavlovic wrote: > >>> James, > >>> > >>> What about stackforge/rally -> openstack/rally ? > >>> > >>> Best regards, > >>> Boris Pavlovic > >> Was the patch for the rename listed here? > >> > >> > https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_Renames > >> > >> This is where we collect our rename patches to schedule them. > >> > >> If you have the patch ready and list it in the above slot in the > >> wikipage we can do the rename with the others in the same maintenance > >> window. > >> > >> Thanks, > >> Anita. > >>> > >>> On Wed, Apr 15, 2015 at 12:57 AM, James E. Blair > >>> wrote: > >>> > >>>> On Friday, April 17 at 22:00 UTC Gerrit will be unavailable for about > 2 > >>>> hours while we rename some projects and perform some database > >>>> maintenance. > >>>> > >>>> Existing reviews, project watches, etc, should all be carried > >>>> over. Currently, we plan on renaming the following projects: > >>>> > >>>> stackforge/murano -> openstack/murano > >>>> stackforge/murano-agent-> openstack/murano-agent > >>>> stackforge/murano-apps -> openstack/murano-apps > >>>> stackforge/murano-dashboard-> openstack/murano-dashboard > >>>> stackforge/murano-deployment -> openstack/murano-deployment > >>>> stackforge/murano-specs-> openstack/murano-specs > >>>> stackforge/python-muranoclient -> openstack/python-muranoclient > >>>> stackforge/murano-docs -> stackforge-attic/murano-docs > >>>> > >>>> stackforge/congress -> openstack/congress > >>>> stackforge/congress-specs-> openstack/congress-specs > >>>> stackforge/python-congressclient -> openstack/python-congressclient > >>>> > >>>> stackforge/tuskar-ui-extras -> stackforge-attic/tuskar-ui-extras > >>>> > >>>> We also plan to transform some database fields in order to support > >>>> Unicode comments. > >>>> > >>>> If you have any questions about the maintenance, please reply here or > >>>> contact us in #openstack-infra on Freenode. > >>>> > >>>> -Jim > >>>> > >>>> ___ > >>>> OpenStack-Infra mailing list > >>>> OpenStack-Infra@lists.openstack.org > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > >>>> > >>> > >>> > >>> > >>> ___ > >>> OpenStack-Infra mailing list > >>> OpenStack-Infra@lists.openstack.org > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > >>> > >> > >> > >> ___ > >> OpenStack-Infra mailing list > >> OpenStack-Infra@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > >> > > > > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] [openstack-dev][all][infra][tc][ptl] Scaling up code review process (subdir cores)
Hi stackers, *Issue* *---* Projects are becoming bigger and bigger overtime. More and more people would like to contribute code and usually core reviewers team can't scale enough. It's very hard to find people that understand full project and have enough time to do code reviews. As a result team is very small under heavy load and many maintainers just get burned out. We have to solve this issue to move forward. *Idea* *--* Let's introduce subsystems cores. Really it's hard to find cores that understand whole project, but it's quite simple to find people that can maintain subsystems of project. *How To* *---* Gerrit is not so simple as it looks and it has really neat features ;) For example we can write own rules about who can put +2 and merge patch based on changes files. We can make special "subdirectory core" ACL group. People from such ACL group will be able to merge changes that touch only files from some specific subdirs. As a result with proper organization of directories in project we can scale up review process without losing quality. *Thoughts?* Best regards, Boris Pavlovic ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Fun (important!) project: optimize Gerrit's nova git repo
Zaro, Thank you for the research, seems like we should definitely run gc against nova repo. Best regards, Boris Pavlovic On Fri, Mar 25, 2016 at 5:47 PM, Zaro wrote: > So I've been researching this and I've found that there is a > significant performance improvement after running git gc on this nova > repro. Below are my results. > > File sizes of repo as-is: > ~/nova.git.orig$ du -hsx * | sort -r | head -10 > 6.4G objects > 6.1M info > 4.0K config > 4.0K HEAD > 382M refs > 2.1M logs > 0B hooks > 0B description > 0B branches > > Note that the repro as-is has already been thru a 'git repack -afd'. > > > File sizes after running 'jgit gc': > ~/nova.git.test$ du -hsx * | sort -r | head -10 > 6.1M packed-refs > 6.1M info > 420M objects > 4.0K config > 4.0K HEAD > 2.1M logs > 0B refs > 0B hooks > 0B description > 0B branches > > The result is that the gc cleans up the objects (6.4G -> 420M) and > moves the loose ref objects from 'refs' dir to a 'packed-refs' file > (382M -> 6.1M). > > Note that I'm using jgit because that's what Gerrit would use to do > the 'gc'. The jgit version is 4.0.1.201506240215-r which is the one > that's packaged with our current version of Gerrit > (2.11.4-11-ga14450f) on review.o.o > > > Here I've tested the performance of the git clone, fetch and push > before and after running 'jgit gc': > > `git clone` > > before: > real 3m30.163s > user 0m2.020s > sys 3m15.087s > > after: > real 0m0.925s > user 0m0.406s > sys 0m0.621s > > > `git fetch origin stable/liberty` > - > before: > real 0m4.271s > user 0m0.701s > sys 0m2.949s > > after: > real 0m0.686s > user 0m0.348s > sys 0m0.307s > > > `git push origin HEAD:refs/for/master` > -- > before: > real 0m36.454s > user 0m5.346s > sys 0m27.598s > > after: > real 0m16.588s > user 0m11.731s > sys 0m3.218s > > Note: I pushed the exact same change for both scenarios. > > > Conclusion: > The results indicate that it would be very advantages to run 'git gc' > for both file size reduction and improved performance. Below are > additional resources that I've found on the internet that seems to > back up my results. > > > > references: > > This says that one-file-per-ref format both wastes storage and hurts > performance: https://git-scm.com/docs/git-pack-refs > > This outlines some of the benefits and drawbacks of packed-refs file: > https://www.mail-archive.com/git%40vger.kernel.org/msg65722.html > > Info on speeding up clones/fetches with pack bitmaps: > https://www.mail-archive.com/git%40vger.kernel.org/msg65571.html > > On Fri, Jan 8, 2016 at 12:13 PM, James E. Blair > wrote: > > Hi, > > > > With the new version of Gerrit offering built-in "git gc" capability, we > > looked at the current state of our git repo maintenance. We run "git > > repack -afd" weekly in an attempt to produce the smallest packfiles > > possible, but it does not prune loose objects, which seems to be the > > main thing "git gc" does that we are missing. > > > > Some (relatively) quick experimentation suggests that various > > combinations of "git gc", "git repack", "git prune", "git prune-packed" > > all have effects on the overall repo size, the number of pack files, and > > the number of loose objects. > > > > However, we don't just want to find the thing that makes the smallest > > repo size (that's easy: "git prune; git gc" -- 394M for nova; one > > packfile with all objects and one packed-refs file with all refs) > > because this repo is used as the basis of all of our mirrors and is > > accessed over several protocols. It's not immediately clear what the > > right optimization is for our situation -- we don't necessarily want to > > trade on-disk size for reduced network performance. Even the packing of > > refs isn't entirely straightforward -- while we haven't needed to for > > some time, we have, in the past removed refs. > > > > We're looking for a volunteer to really dig into this problem and > > thoroughly evaluate the implications of different ways of optimizing the > > repo. If you're interested, you can download a snapshot of the full > > nova repository from Gerrit (it is a point-in-time snapshot and will not > > be updated) at this U
