Brice Figureau <brice-pup...@daysofwonder.com> writes: >> Our mysql questions, com_select and com_insert stats spike first. >> com_select and com_update are normally at around 5 and spike to 40; >> questions is normally around 150 and spikes to 600. Threads connected >> goes from around 15 to 30. After that it looks like everything queues >> up behind MySQL and we start getting timeouts on our ~450 clients.
... > The only reason for a storeconfig storm is that Puppet deletes all the > resources/tags belonging to a particular host and then recreates them, > so you see a lots of Inserts. > > Now the real question is why Puppet thinks there is such discrepencies > between the database and the live compilation. > > Are you sure you're not removing hosts from the database? Yes. Hosts that no longer exist are still in the database :) > What would be interesting is to activate the mysql general query log > (warning it will increase your load), and dig in the large log around > the timeframe you see the storm (you can also activate the rails log for > the same effect). I meant to ask whether some MySQL expert could look at our binary logs and figure out what happened :) Oh, I see there's a mysqlbinlog command! Who knew? Some quick greps of its output say the total number of updates and inserts from yesterday is about the same as any other day. Same for various hours yesterday--the hour that we got slammed doesn't seem to have more updates or inserts than other hours when we didn't get slammed. > Or I remember reading that maatkit now contains a query log extractor > from tcpdump captures files; it is worth capturing the traffic between > Puppet and mysql and analyze the queries performed. Maybe you'll find > the issue. Maybe there's a tool which reads binary logs and tells you what caused the storm :) > Good luck :-) Heh, thanks :) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
