On 05/16/2014 09:29 AM, Johan Sunnerstig wrote: > Hello, > > I'm just wondering how people with a decent collection of modules are > organizing their modules and repositories? > We currently have one big repository with everything in it, and three > branches, basically stuff moves alpha -> beta -> production. This is > nice and simple in a way, but has some obvious and not so obvious downsides. > > For one updating a module means going to the "next" branch and doing a > checkout of that specific module from the previous branch so as to not > just yank in every change made in the previous branch. This along with > having just one big commit history for everything makes the git history > pretty much useless. > And we're now in the process of implementing Jenkins for CI, the lack of > granularity is making this a bit of a headache(though this could very > well be because we're complete newbies at Jenkins ;) . For example > having a post-receive hook triggering a build in Jenkins, with one big > repo this will trigger rebuilds of every module since they all subscribe > to the same repo even though they're separate jobs. > > So, how are you people who have actually done this for a while managed > this little problem? Separate repos per module? Something else we > haven't thought of? > > Regards > Johan >
Johan,
I actually have 2 unique workflows between work and personal puppet
environments. In both cases they utilize Jenkins CI and git repos to
manage but in different ways. That said it appears that our workflow for
work is going to be moving more towards how I currently manage my own
puppet environment.
I will leave open for discussion if this is considered a best practice
within the community but this is how I'm managing it and I keep it out
there for anyone to make use of as starting point. I have my module
repository available at https://github.com/UGNS/standard-modules and I'm
currently running with puppet 3.5.1 though I'm working to update it for
3.6.0 and using directory environments which it's nearly ready for as is.
I make use of r10k to deploy so my repo contains the manifests/
sub-directory along with the Puppetfile which r10k uses to deploy out to
the modules/ sub-directory. I plan to obviously add an environment.conf
for 3.6.0+ directory environment support. With r10k every module is
either pulled from the Forge or a git repo. This allows the modules to
be developed independently of what is deployed out for a given environment.
With r10k each branch is itself a separate environment. Currently I
operate with 3 branches (development, staging & production) which my
Jenkins CI jobs handle. I have the GitHub repo setup using Amazon SQS
which Jenkins is setup as a consumer and triggers the job runs when
commits are pushed to the repo. The job uses librarian-puppet locally on
the Jenkins host to process the Puppetfile and verify that it doesn't
attempt to install any other dependencies than what is defined in the
Puppetfile itself. Upon success it triggers a job to execute r10k over
an SSH connection to the puppet master followed by triggered job to
execute an 'mco puppet runall <CONCURRENCY> -F environment=<BRANCH>' to
force execution on all hosts within that environment.
Successful job runs can then be approved for promotion which runs the
triggered job to merge from one branch to the other (development ->
staging, staging -> production) and promoted to production I tag it with
the date/time stamp.
I still have some thoughts on how better to improve what I have but
haven't had time to develop them yet. I'd like to make the process more
fluid within Jenkins so it would allow for feature branches to use as
test environments that could then be merged in rather than having a
fairly strict configuration that only supports the 3 existing branches
and ignores everything else. I'm also looking at the option of using mco
itself to trigger the r10k deployment rather than having to do so over
an SSH connection. Along with the fact the system can't easily stand
itself up without being done in stages currently.
signature.asc
Description: OpenPGP digital signature
