When a stash is being performed on a non-master agent, is the stash
operation really CPU-intensive on the master? I wasn't sure from that
description whether it assumed the master was also executing the stash
phase. We did try tripling the CPU allocated to the master's VM, and saw
no change in stash performance.
As recently as three weeks ago, this same stash pipeline stage on the
same repository was executing in 30-45 seconds. The repo's only grown
fractionally since then. If I'm at the limit of the stash
implementation, I hit a heck of a wall.
Thanks,
-nat
On Thu, Dec 14, 2017, at 03:58 PM, Mark Waite wrote:
>
>
> On Thu, Dec 14, 2017 at 4:13 PM Nathaniel Irons
> <na...@bumppo.net> wrote:>> Jenkins is building an iOS project with five
> similarly-sized targets.>> The git repository is 251 MB, of which the .git
> dir is 130 MB. A
>> pipeline stage handles the checkout, injects build numbers, and
>> stashes>> the modified sources, minus the unnecessary .git dir, for the
>> other>> build nodes to consume.
>>
>> stage ("Checkout and modify") {
>> agent { label 'xcode-9' }
>> steps {
>> checkout scm
>> sh ("agvtool new-version -all $BUILD_ID")
>> stash (excludes: '.git', name: "modified-sources")
>> }
>> }
>>
>> This stash operation consistently takes 15 minutes. Unstashing it
>> on any>> of the build nodes then takes about ten minutes per node.
>> Something is>> wrong, but I'm not sure what steps to take.
>>
>
> Unfortunately, I don't have a solution to offer.
>
> The online help for stash (in the Pipeline syntax when "Stash - Stash
> some files to be used later in the build" is selected), hints that you
> may be at the limits of the stash implementation. It says:>
> Note that the stash and unstash steps are designed for use with small
> files. For large data transfers, use the External Workspace Manager
> plugin, or use an external repository manager such as Nexus or
> Artifactory. This is because stashed files are archived in a
> compressed TAR, and with large files this demands considerable on-
> master resources, particularly CPU time. There's not a hard stash size
> limit, but between 5-100 MB you should probably consider alternatives.>
> Mark Waite
>
>
>> The build environment is vsphere on an older mac pro, with some
>> Mac VMs>> for Xcode and a (dockerized) jenkins master in a linux VM. The
>> storage>> is all SSD, and we're not seeing any issues with I/O
>> performance other>> than this stash/unstash problem. Zipping this same
>> source directory>> takes six seconds, and tar+bzip2 takes 15 seconds.
>>
>> An individual build of this project runs in under three minutes,
>> so the>> unstash alone quadruples our build times. Any suggestions for how
>> to>> investigate?
>>
>> Thanks,
>>
>> -nat
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Jenkins Users" group.>> To unsubscribe from this group and stop
>> receiving emails from it,
>> send an email to jenkinsci-users+unsubscr...@googlegroups.com[1].>> To
>> view this discussion on the web visit
>>
>> https://groups.google.com/d/msgid/jenkinsci-users/1513293192.3586726.1205509440.0B960302%40webmail.messagingengine.com.>>
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.> To unsubscribe from this group and stop
> receiving emails from it,
> send an email to jenkinsci-users+unsubscr...@googlegroups.com.> To view
> this discussion on the web visit
>
> https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtE0k5MswmNzRhTRfcHCdq3Ry%2BXbognyX6km1GQ_c8-eEw%40mail.gmail.com[2].>
> For more options, visit https://groups.google.com/d/optout.
Links:
1. mailto:jenkinsci-users%2bunsubscr...@googlegroups.com
2.
https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtE0k5MswmNzRhTRfcHCdq3Ry%2BXbognyX6km1GQ_c8-eEw%40mail.gmail.com?utm_medium=email&utm_source=footer
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/1513302770.3633224.1205623960.21FCC9DC%40webmail.messagingengine.com.
For more options, visit https://groups.google.com/d/optout.
