There actually is a way to do this, though you may find it to be more
painful to work with.
Imagine, if you will, two environments: production and maintenance.
The production environment is the one you're running right now, for
production. It fully manages everything and ensures that your systems are
all fully up-to-spec. It takes about 5 minutes for a full run of this
manifest.
The maintenance environment, on the other hand, manages /etc/passwd,
exported resources, and a couple critical resources that change frequently.
It doesn't check package versions, update /etc/ssh/ssh_known_hosts,
configure backup software, etc. It's main purpose is to keep puppet
running.
Once you have these two environment configured, you move the majority of
your hosts from "production" to "maintenance", and your puppet runtime
drops. When you make actual changes to the manifests, you temporarily move
all those hosts back into the production manifest so they get applied, and
then revert them to maintenance.
Another possibility for reducing overall CPU usage is to reduce the number
of times a day that Puppet runs. If you cut it back to twice daily, then
your total CPU usage goes from 120 minutes/host to 10 minutes/host. That
is, in fact, how we run Puppet where I work, though we do that out of a
culture of a "no changes during production" mindset rather than saving CPU
cycles.
Finally, consider the actual reasons for your long run times. If it's
primarily that you are checksumming large file trees, you may want to
consider other alternatives. While Puppet is fabulous for templated files,
perhaps the bulk of those files could go into a bzr/svn/git/hg/whatever
repository? Then your manifest for that directory is reduced to an exec{}
for creating it, and either an exec{} or perhaps a cron{} for running the
appropriate update.
On Tue, Jun 19, 2012 at 6:38 AM, jcbollinger <john.bollin...@stjude.org>wrote:
>
>
> On Tuesday, June 19, 2012 5:23:42 AM UTC-5, Duncan wrote:
>>
>> Hi folks, I'm scratching my head with a problem with system load.
>>
>> When Puppet checks in every hour, runs through all our checks, then exits
>> having confirmed that everything is indeed as expected, the vast majority
>> of the time no changes are made. But we still load our systems with this
>> work every hour just to make sure. Our current configuration isn't perhaps
>> the most streamlined, taking 5 minutes for a run.
>>
>> The nature of our system, however, is highly virtualised with hundreds of
>> servers running on a handful of physical hosts. It got me thinking about
>> how to reduce the system load of Puppet runs as much as possible.
>> Especially when there may be a move to outsource to virtualisation hosts
>> who charge per CPU usage (but that's a business decision, not mine).
>>
>> Is there a prescribed method for reducing Puppet runs to only be done
>> when necessary? Running an md5sum comparison on a file every hour isn't
>> much CPU work, but can it be configured so that Puppet runs are triggered
>> by file changes? I know inotify can help me here, but I was wondering if
>> there's anything already built-in?
>>
>
> You seem to be asking whether there's a way to make the Puppet agent run
> to see whether it should run. Both "no, obviously not" and "yes, it's
> automatic" can be construed as correct answers. In a broader context,
> anything you run to perform the kind of monitoring you suggest will consume
> CPU. You'd have to test to see whether there was a net improvement.
>
> Consider also that although file checksumming is one of the more expensive
> operations Puppet performs, files are not the only managed resources in
> most Puppet setups. You'll need to evaluate whether it meets your needs to
> manage anything only when some file changes.
>
> There are things you can do to reduce Puppet's CPU usage, however. Here
> are some of them:
>
> - You can lengthen the interval between runs (more than you already
> have done).
> - You can apply a lighter-weight file checksum method (md5lite or even
> mtime).
> - You can employ schedules to reduce the frequency at which less
> important resources are managed.
> - You can minimize the number of resources managed on each node.
>
> John
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/5UkHTsXNKIsJ.
>
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
