[JIRA] (JENKINS-44085) have a simple way to limit the number of parallel branches that run concurrently

[sam.mxra...@gmail.com (JIRA)]

Title: Message Title

Sam Gleske commented on JENKINS-44085   Re: have a simple way to limit the number of parallel branches that run concurrently Added lockable-resources-plugin as a potential dependency since a semaphore step could be implemented. Semaphore-like behavior with lock step Can be achieved by calculating lock names using modulo operator to cycle through an integer. Here's an example using rainbow colors. int concurrency = 3 List colors = ['red', 'orange', 'yellow', 'green', 'blue', 'indigo', 'violet'] Map tasks = [failFast: false] for(int i=0; i<colors.size(); i++) { String color = colors[i] int lock_id = i % concurrency tasks["Code ${color}"] = { -> stage("Code ${color}") { lock("color-lock-${lock_id}") { echo "This color is ${color}" sleep 30 } } } } // execute the tasks in parallel with concurrency limits stage("Rainbow") { parallel(tasks) } The above code will execute 7 stages in parallel; however it will not run more than 3 concurrently. The above will create custom locks: color-lock-0 color-lock-1 color-lock-2 All concurrent tasks will race for one of the three locks. It's not perfectly efficient (certainly not as efficient as a real semaphore) and there are some limitations. Limitations with this workaround Your pipeline will take as long as your slowest locks. So if you unfortunately have several long running jobs racing for the same lock (e.g. color-lock-1), then your pipeline could be longer than if it were a proper semaphore. Example scenario with the three locks: color-lock-0 takes 20 seconds to cycle through all jobs. color-lock-1 takes 30 minutes to cycle through all jobs. color-lock-2 takes 2 minutes to cycle through all jobs. Then your job will take 30 minutes to run... where as with a true semaphore it would have been much faster because the longer running jobs would take the next available lock in the semaphore rather than be blocked. Add Comment   This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.181710.1494001199000.5885.1574568121112%40Atlassian.JIRA.