On Mon, Aug 4, 2014 at 10:11 AM, Brian Moffat <crd...@gmail.com> wrote:
> I have a matrix project to build different C# projects/solutions. There > is one git repository for all solutions. > > The matrix job is configured for Git to checkout the master branch to a > local directory, and then invokes a downstream project for each matrix > element (solution). That downstream project has not Git configuration, and > simply builds the solution using MSBuild. > > What I am observing (looking at the console output for the Matrix build > job instance) is the matrix job performs the git checkout, which takes > about 4 minutes. It then says "Triggering <matrix element>", but that > downstream project doesn't show up in the build list for another 4-5 > minutes. Based on noticing that the content of the checkout directory is > changing (total size differences) during the period between when it says > "Triggering ..." and when that downstream job actually starts, my guess is > that another git checkout is occurring behind the scenes. > Your guess is correct. Each instance gets a full copy of the repository. If your repository is large, you could add the "Additional behaviours" for "Advanced clone options" and use a reference repository on the build machine(s). For example, if you created a bare repository copy of your central git repo on each build machine at c:\bare\your-git-repo.git, then you could list c:\bare\your-git-repo.git as the reference repository location. Git will then reference the contents of that bare repository and use that to reduce the "fetch" time. That won't reduce the "checkout" time, just the "fetch" time. If you also need to reduce the checkout time, then you might consider using "Sparse checkout" to only checkout those subdirectories which are actually required for your build. Another alternative would be to implement "git-work-dir" support in the git plugin, and submit a pull request for it. Refer to https://issues.jenkins-ci.org/browse/JENKINS-23594 for an enhancement request asking for that support. That enhancement request seems to match your use case quite well. > Obviously, this adds a significant amount of time to the total length of > the matrix build. The builds of each solution take under 1 minute. So for > the 7 solutions, I would expect the length of time to be 4+(7*1) or roughly > 11 minutes, but with the 4-5 minute delay between each matrix element, it > takes (4*7) + (7*1) or roughly 30 minutes. Our CI/CD processes are > maturing, and the lengthy delay between checkin at deploy to staging is now > slowing us down. > > Can anyone provide me guidance on what is going on behind the scenes > (nothing in any Jenkins log that I can find), or tell me how I can > configure the matrix job "properly" to not do this? > > This has run fast, sometimes - but I think perhaps it stopped working > after updating plugins, or may somehow be related to the plugins installed > - I just don't know where to start to look. > > The git section of the matrix job has: > - Additional Behaviors : Checkout to a subdirectory: > C:\Build\Source\WebPortal\Master\Dev > - Advanced sub-module behavior: Recursively update submodules (checked) > - Prune stale remote-tracking branches (not sure this is necessary?) > > Here is the console log for the matrix job: > > Started by user < <http://autobuild.blueport.com/user/bmoffat>me> > [EnvInject] - Loading node environment variables. > Building in workspace C:\Tools\Jenkins\jobs\BuildAllWebPortals\workspace > > C:\Tools\Git\cmd\git.exe rev-parse --is-inside-work-tree > Fetching changes from the remote Git repository > > C:\Tools\Git\cmd\git.exe config remote.origin.url <our git repository url> > > Pruning obsolete local branches > Fetching upstream changes from <our git repository url> > > > C:\Tools\Git\cmd\git.exe --version > > C:\Tools\Git\cmd\git.exe fetch --tags --progress <our git repository url> > +refs/heads/master:refs/remotes/origin/master --prune > > C:\Tools\Git\cmd\git.exe rev-parse "origin/master^{commit}" > Checking out Revision b18aa8cff0aa9da9166f740877845cef27b612ff (origin/master) > > C:\Tools\Git\cmd\git.exe config core.sparsecheckout > > C:\Tools\Git\cmd\git.exe checkout -f > b18aa8cff0aa9da9166f740877845cef27b612ff > > C:\Tools\Git\cmd\git.exe rev-list b18aa8cff0aa9da9166f740877845cef27b612ff > > C:\Tools\Git\cmd\git.exe remote > > C:\Tools\Git\cmd\git.exe submodule init > > C:\Tools\Git\cmd\git.exe submodule sync > > C:\Tools\Git\cmd\git.exe config --get remote.origin.url > > C:\Tools\Git\cmd\git.exe submodule update --init --recursive > Email was triggered for: Before Build > Sending email for trigger: Before Build > Sending email to: <distribution list> > Triggering MobileASI > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=MobileASI/> > > MobileASI > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=MobileASI/> > completed with result SUCCESS > Triggering Furniture > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=Furniture/> > > Furniture > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=Furniture/> > completed with result SUCCESS > Triggering RetailPortal > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=RetailPortal/> > > RetailPortal > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=RetailPortal/> > completed with result SUCCESS > Triggering MobileLeons > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=MobileLeons/> > > MobileLeons > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=MobileLeons/> > completed with result SUCCESS > Triggering Leons > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=Leons/> > > Leons <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=Leons/> > completed with result SUCCESS > Triggering MobileVCF > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=MobileVCF/> > > MobileVCF > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=MobileVCF/> > completed with result SUCCESS > Triggering RetailTool > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=RetailTool/> > > RetailTool > <http://autobuild.blueport.com/job/BuildAllWebPortals/Portal=RetailTool/> > completed with result SUCCESS > No emails were triggered. > Finished: SUCCESS > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Thanks! Mark Waite -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
