On Tue, 2008-10-14 at 10:22 -0700, Paul Lathrop wrote: > On Fri, Oct 10, 2008 at 4:39 AM, Kenneth Holter <[EMAIL PROTECTED]> wrote: > > > > Hello list. > > > > > > We're running puppet version 0.24.4, and are having trouble with > > puppetmaster running really slow. The puppetmaster has been running fine > > until about a couple of day ago, and for no apparent reason it just lost > > performance. I've also noticed that the puppet daemon on many of our clients > > a dying. Both master and clients run very slow, but I'm not sure if the > > client runs slow because of the master. > > > > The symptoms I've seen are that running puppet on a client often takes a > > very long time (one of the nodes used close to 800 seconds to run the > > config), or even times out. Once in a while, however, things seems back to > > normal, and the run is executed in a few seconds. On the server side the > > time between "Compiled configuration for ..." messages takes a very long > > time (often several minutes), in contrast to normal operations where > > configurations get compiled quite frequently. > > > > What puzzles me is that there are no signs of resource starvation on the > > server - we have low CPU and memory utilization. So there shouldn't be, as > > far as I can see, any hardware limitations causing puppetmaster to run slow. > > > > I don't have the exact number of active puppet nodes, but it should be well > > under 100 nodes. > > > > Other than that, I can mention that "storeconfig" is set to "true" > > in puppet.conf. > > <snip> > > This is the telling point for me. I ran into the same problem myself. > The storeconfig database has some terrible issues. I wish I could tell > you how to solve this; we ended up just abandoning stored configs. > Others have had better luck. > > One thing you might try is removing stale entries from your stored > config database. There's a script floating around out there...
0.24.6 will contain (actually 0.24.6-rc1 contains) a patch that fixes several performance issues with storeconfig. Peter Meier even reported that it reduced its master (or sql) load by half : http://groups.google.com/group/puppet-dev/browse_thread/thread/6a74e8103eff4313/363d90830880690c#363d90830880690c My test show that on my master, catalog compilation for a 500 resources node takes now 6s instead of 15s before my patch (storeconfig was about 60% of the compilation time). The complete information about the patch is here: http://projects.reductivelabs.com/issues/show/1502 I also submitted a patch to refine the current db schema which uses multiple unused index (it is also part of 0.24.6-rc1). You need to migrate the schema with --dbmigrate: http://projects.reductivelabs.com/issues/show/1506 -- Brice Figureau <[EMAIL PROTECTED]> --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
