On Tue, 2009-07-28 at 08:50 -0400, Mark Plaksin wrote: > Howdy: > > Does anybody else see in storeconfigs spikes *after* you've been up and > running with storeconfigs for a while? Twice in the past month our > puppetmaster has been slammed by storeconfigs activity. We're running > 25b2 but not (yet) puppetqd. > > Our mysql questions, com_select and com_insert stats spike first. > com_select and com_update are normally at around 5 and spike to 40; > questions is normally around 150 and spikes to 600. Threads connected > goes from around 15 to 30. After that it looks like everything queues > up behind MySQL and we start getting timeouts on our ~450 clients. > > The storm lasts less than an hour. > > Of course nothing special is going on with our clients (that we know > of!) when the storm hits. I *think* but am not positive that our first > storm happened while we were running mod_proxy + Mongrel. Our second > happened with Passenger.
The only reason for a storeconfig storm is that Puppet deletes all the resources/tags belonging to a particular host and then recreates them, so you see a lots of Inserts. Now the real question is why Puppet thinks there is such discrepencies between the database and the live compilation. Are you sure you're not removing hosts from the database? What would be interesting is to activate the mysql general query log (warning it will increase your load), and dig in the large log around the timeframe you see the storm (you can also activate the rails log for the same effect). Or I remember reading that maatkit now contains a query log extractor from tcpdump captures files; it is worth capturing the traffic between Puppet and mysql and analyze the queries performed. Maybe you'll find the issue. Good luck :-) -- Brice Figureau My Blog: http://www.masterzen.fr/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
