We’ve had a number of problems with using clone-workspace-plugin in a
job pipeline.
The two problematic issues we have are:
(1) Poor archiving performance
(2) It only maintains a clone on the very latest instance of the job
at the top of the pipeline
Our build is somewhat monolithic by some standards. We have all
required libraries checked in alongside our source code, so a built
workspace is around 1.5Gb. clone-workspace-plugin (or the
combination of it and the slave.jar mechanism) takes around 50 minutes
to archive this to the master at the end of the job and a similarly
long time to un-zip it at the start of each downstream job.
Partly because of this delay, (2) comes in to play, there is the
inevitable race condition:-
The pipeline is (a) compile (b) package (c) quicktest (d) fulltest.
Once (a) has completed workspace.zip is created for that changelist
(lets call it @000001).
(b) kicks off and usually gets the same workspace.zip.
Meanwhile a developer submits another change, and another (a) is
starts (call this changelist @000002).
At the end of (a)@000002 that a new workspace.zip is created perhaps
before (c)@000001 began.
(c) then starts, it is supposed to be (c)@000001 but it actually gets
the workspace from (a)@000002.
The displayed change history is correct but now not possible to tell
whether any test regressions were caused by @000001 or @000002 because
we have no test results for the former.
A bit later, (b)@000002 completes and triggers the real (c)@000002.
This is being run needlessly.
Late in the afternoon on a busy day it's not unusual to have half a
dozen (d) jobs queued up, all triggered from different (a) changelists
but they will all actually run an identical changelist from the most
recent (a).
To this end I wrote a script to replace this plugin, it uses $
{BUILD_TAG} to ensure a separate archive is created for each starting
changelist.
This works very well. It takes 2-3 minutes to create and unpack the
archive, faster by 10-30x.
The script is attached with more details on the wiki page
https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin
and https://issues.jenkins-ci.org/browse/JENKINS-10520
We still use clone-workspace-plugin (archiving a single file which is
quick) in order to be able to display the perforce change history -
but due to the race condition this is sometimes wrong.
We'd like it to display the upstream change history of the exact job
that triggered the pipeline, not the latest one.
Regards, Ed.
