> > I still wonder why there are no options out-of-the-box to: Probably noone needed it badly enough to do it or to pay for it :)
It seems to me that in the Git world, several smaller repos are preferred instead of a big monolithic one. Not sure about you, but in our case the big repo was inherited from a subversion setup. When switching to Git, we kept the centralised workflow and its big repo with lots of obsolete branches, but since then there has been a natural tendency to keep new things in separate repos. While this story may be common in the corporate world, it's probably less so in the more modern open source/startup/smaller agile projects - which I guess are more of a driving force for Jenkins.. Also, while the problem has annoyed me for a long time, it hasn't been a blocker so that I haven't even tried this workaround until recently.. The other reason may be that this would basically involve auto-creating jobs (or something equivalent) on each slave, managed from a central location... at a first glance this differs from the standard ways plugins extending Jenkins. Anyway If I had time I'd have a go at building a plugin - unless anyone started already? On 16 December 2013 20:16, David Gayman <qbd...@vt.edu> wrote: > Greg, > > Thank you for responding, I am somewhat new to git and had no idea about > the --reference option. Your solution looks like basically the best way to > do this. > > I still wonder why there are no options out-of-the-box to: > > 1) Set up only one cloned repository per project, and > 2) Refuse to check out repositories at the top level (on the machine which > is conducting the build) > > Or, if there are options to do this but I don't know where they are. It > seems like the setup you and I have should be the default! The only thing I > figure is that for generality's sake, each build should have a unique clone > of the repo. > > Thanks. > > *Also, my terminology is too loose - I meant "clone" not "checkout". > > > > > > On Mon, Dec 16, 2013 at 12:59 PM, Gergely Nagy <gsz...@gmail.com> wrote: > >> Not sure how have the exact same "checkout" multiple times is useful >> (most of my jobs modify the workspace so I need to isolate them from each >> other -> separate workspace == separate "checkouts"). >> >> OTOH, I was also interested to save on disk space when it comes to large >> git repos cloned multiple times. Note that I mean "clone" not "checkout". >> >> So, an easy/safe enough solution to reduce waste of duplicate clones: >> 1) have 1 proper clone of the big repo on each slave - just by create a >> dumb "mirror" job with no build steps, just to keep the clone up to date >> 2) the actual jobs use the "Reference" repo option: although the main >> clone location can be still be the central server, you point to that >> workspace of the mirror job as a reference (see --reference >> https://www.kernel.org/pub/software/scm/git/docs/git-clone.html, and the >> Advanced git plugin option). >> >> My results using this is quite dramatic, the repo (.git) sizes under the >> worker jobs are just a few Mb, even the actually repo is 1.5G+ in my case. >> Also the "clone" for each worker job will just take a fraction of time. >> >> Now I'd like Jenkins (plugin?) to provide a setup like this >> out-of-the-box, eliminating the manual fiddling (I needed to add site-wide >> variables overridden for each slave, etc..). Anyway it's good enough for me >> now. >> >> Of course the actual "checkout" sizes are the same, but I see no >> safe&straightforward way around that, that's just the nature of >> building/testing code. >> So, not sure if this helps but good luck anyway. >> Greg >> >> >> >> On 16 December 2013 17:06, <qbd...@vt.edu> wrote: >> >>> Has anyone considered a modification to the git plugin (or other SCM >>> plugins), to reduce the total number of checkouts physically required on a >>> slave machine? >>> >>> We run 6 mult-configuration projects (debug and release configurations) >>> on a handful of computers. Each configuration requires one code checkout >>> (2.1 G). That's 2.1*2*6 = 25.2 G on each slave machine, for the source >>> code only. The master build machine requires another 12.6 G, because it >>> checks out additional copies *outside* the scope of build directories (no >>> one knows why). Forget about leaving "Advanced Project Options" > "Restrict >>> where this project can be run" unchecked, because if the build is started >>> on a different computer each time, the code has to be checked out at the >>> top level (potentially adding the 12.6 G to each slave machine, if the load >>> is shared). >>> >>> Perhaps these issues have already been addressed; we run Jenkins version >>> 1.524. >>> >>> Suggestions: >>> a) Live with redundant code checkouts >>> b) Stop using the git plugin, and have my shell script handle code >>> checkout >>> c) Rewrite the git plugin, or add an option, to allow one checkout per >>> project on a slave. Technically this is the minimum number of checkouts >>> allowed, because each project is allowed to access a unique sha1 hash. >>> d) Add an option to the Jenkins core to disregard the top-level code >>> checkouts (which I gather are done for some kind of consistency check...we >>> can live without that check I think). >>> e) I don't know Java and don't have much free time, otherwise I would >>> write a completely new git plugin. >>> >>> Thanks for any suggestions, especially if I am missing something obvious >>> here. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to jenkinsci-users+unsubscr...@googlegroups.com. >>> >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Jenkins Users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/jenkinsci-users/IwgQSKqrC6g/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> jenkinsci-users+unsubscr...@googlegroups.com. >> >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
