One quick comment is that in our environment, we have found that having jenkins poll the source control system is extremely unreliable. We switched to using a commit hook (post-receive after we switched from svn to git) that fires off a specific job on our various Jenkins masters, passing in the branch affected and the list of changed files. This jenkins job then evaluates the changeset and triggers specific downstream jobs (or ignores it, depending).
On Wed, Apr 9, 2014 at 7:35 AM, Frank Roghair <frank.rogh...@gmail.com>wrote: > We are the System & Verification team from Philips Healthcare and are > responsible for the implementation and support of a multisite Build and > Test Infrastructure. This infrastructure supports continuous integration. > > > > A year ago we brought our continuous integration service under control of > Jenkins, see next diagram. > > > > > <https://lh4.googleusercontent.com/-w_-L6M7gmDs/U0VZJyDbuHI/AAAAAAAAABQ/Gcugmxg4V5E/s1600/jenkins_setup.png> > > > > This resulted in more than 1000 Jenkins jobs, structured into Jenkins flow > control jobs. The trigger for such a flow is a check in by a developer, > which Jenkins recognizes by polling the Version Control System. Every > further step in the flow is under control of Jenkins. > > > > Characteristics of our continuous integration service : > > # of projects: > > 30+ > > # of jobs: > > 1000+, structured in flows > > # of flows : > > 30+ > > # of systems: > > 200+ > > > > We are faced with the following Jenkins scaling challenges: > > · Thundering herds clogging Jenkins, downstream systems, e.g. > changing the tooling will trigger all Jenkins jobs. These will start all > together in parallel, what is unwanted. We like to do this in a more > controlled way, e.g. by defining priorities. > > · Making global changes across jobs > > · Making sure plugins can handle 1000+ jobs > > · Testing new Jenkins and plugin versions > > · Making consistent backups > > > > We like to share our experiences with other groups but are especially > looking for solutions for the Jenkins scaling issues. We like to get > contact with groups that had the same type of scaling issues and found > solutions for that. > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Marc MacIntyre -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
