On Thu, Dec 14, 2017 at 6:53 PM Nathaniel Irons <na...@bumppo.net> wrote:
> When a stash is being performed on a non-master agent, is the stash
> operation really CPU-intensive on the master? I wasn't sure from that
> description whether it assumed the master was also executing the stash
> phase. We did try tripling the CPU allocated to the master's VM, and saw no
> change in stash performance.
>
> As recently as three weeks ago, this same stash pipeline stage on the same
> repository was executing in 30-45 seconds. The repo's only grown
> fractionally since then. If I'm at the limit of the stash implementation, I
> hit a heck of a wall.
>
>
That sounds like something else has changed. If it was working well
before, and has only grown a little since then, it seems like it is worth
investigating other things that might have changed.
Has the disc I/O capability on the master or the agent changed
significantly?
If you perform a smaller stash from the same workspace, does it perform
significantly better?
Are there competing processes which might be slowing throughput?
Has the Jenkins memory use increased significantly?
Mark Waite
> Thanks,
>
> -nat
>
>
> On Thu, Dec 14, 2017, at 03:58 PM, Mark Waite wrote:
>
>
>
> On Thu, Dec 14, 2017 at 4:13 PM Nathaniel Irons <na...@bumppo.net> wrote:
>
> Jenkins is building an iOS project with five similarly-sized targets.
> The git repository is 251 MB, of which the .git dir is 130 MB. A
> pipeline stage handles the checkout, injects build numbers, and stashes
> the modified sources, minus the unnecessary .git dir, for the other
> build nodes to consume.
>
> stage ("Checkout and modify") {
> agent { label 'xcode-9' }
> steps {
> checkout scm
> sh ("agvtool new-version -all $BUILD_ID")
> stash (excludes: '.git', name: "modified-sources")
> }
> }
>
> This stash operation consistently takes 15 minutes. Unstashing it on any
> of the build nodes then takes about ten minutes per node. Something is
> wrong, but I'm not sure what steps to take.
>
>
> Unfortunately, I don't have a solution to offer.
>
> The online help for stash (in the Pipeline syntax when "Stash - Stash some
> files to be used later in the build" is selected), hints that you may be at
> the limits of the stash implementation. It says:
>
> Note that the stash and unstash steps are designed for use with small
> files. For large data transfers, use the External Workspace Manager plugin,
> or use an external repository manager such as Nexus or Artifactory. This is
> because stashed files are archived in a compressed TAR, and with large
> files this demands considerable on-master resources, particularly CPU time.
> There's not a hard stash size limit, but between 5-100 MB you should
> probably consider alternatives.
>
> Mark Waite
>
>
>
> The build environment is vsphere on an older mac pro, with some Mac VMs
> for Xcode and a (dockerized) jenkins master in a linux VM. The storage
> is all SSD, and we're not seeing any issues with I/O performance other
> than this stash/unstash problem. Zipping this same source directory
> takes six seconds, and tar+bzip2 takes 15 seconds.
>
> An individual build of this project runs in under three minutes, so the
> unstash alone quadruples our build times. Any suggestions for how to
> investigate?
>
> Thanks,
>
> -nat
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/1513293192.3586726.1205509440.0B960302%40webmail.messagingengine.com
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtE0k5MswmNzRhTRfcHCdq3Ry%2BXbognyX6km1GQ_c8-eEw%40mail.gmail.com
> <https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtE0k5MswmNzRhTRfcHCdq3Ry%2BXbognyX6km1GQ_c8-eEw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/1513302770.3633224.1205623960.21FCC9DC%40webmail.messagingengine.com
> <https://groups.google.com/d/msgid/jenkinsci-users/1513302770.3633224.1205623960.21FCC9DC%40webmail.messagingengine.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtF3ueKhOO3CtmU1Tb7VxnbTMRJRAZZcVn%2BoKOLrTXYaTw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
