On Mon, Jul 23, 2018 at 8:26 AM Allen Wittenauer <a...@effectivemachines.com.invalid> wrote:
> > > On Jul 23, 2018, at 12:45 AM, Gavin McDonald <ga...@16degrees.com.au> > wrote: > > > > Is there any reason at all to keep the 'workspace' dirs of builds on the > > jenkins slaves ? > > Yes. > > - Some jobs download and build external dependencies, using the > workspace directories as a cache and to avoid sending more work to INFRA. > Removing the cache may greatly increase build time, network bandwidth, and > potentially increase INFRA’s workload. > +1, also occasionally there are network flakes downloading dependencies and when we were using Maven we were unable to find a way to get it to retry dependency downloads so this would routinely fail the build. Not wiping out the workspace between builds was the only way to get the build to be remotely consistent. Using a per-buildslave repository instead of a per-workspace repository was likewise unworkable because it multiplied the chance of the repository getting into a bad state by the number of different types of builds happening on the machine. > > - This will GREATLY greatly increase pressure on the source > repositories, as every job will now do a full git clone/svn checkout. > Hadoop’s repo size just passed 700M. > > - Many jobs don’t put everything into the saved artifacts due to > size constraints. Removing the workspace will almost certainly guarantee > that artifact usage goes way up as the need to grab (or cache) bits from > the workspace will be impossible with an overly aggressive workspace > deletion policy. > > Given how slow IO is on the Windows build hosts, this list is > especially critical on them. > > > And , in advance, I'd like to state that projects creating their own > > storage area for jars and other artifacts to quicken up their builds is > not a valid reason. > > Maven, ant, etc don’t perform directory locks on local > repositories. Separate storage areas for jars are key so that multiple > executors don’t step all over each other. This was a HUGE problem for a > lot of jobs when multiple executors were introduced a few years ago. -- ------- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback
