Well after several attempts at tuning the DB config and puppetdb config, we had to drop the postgresql database and recreate it then allowing puppetdb to create the required tables, indexes, etc. Now the command queue is going between zero and four, processed tens of thousands of queue commands (since restarting at 10:15am) and the cpu load on the server is next to nothing.
My guess is the database was corrupt somehow with all of the deletes from the KahaDB directory plus trying to remove a huge amount of facts. I think puppet needs to provide some type of DB reset process without have to drop/create the DB. On Wednesday, June 28, 2017 at 12:25:57 PM UTC-4, Peter Krawetzky wrote: > > Last Sunday we hit a wall on our 3.0.2 puppetdb server. The cpu spiked > and the KahaDB logs started to grow eventually almost filling a > filesystem. I stopped the service, removed the mq directory per a > troubleshooting guide, and restarted. After several minutes the same > symptoms began again and I have not been able to come up with a puppetdb or > postgresql config to fix this. > > We tried turning off storeconfig in the puppet.conf file on our puppet > master servers but that doesn't appear to have resolved the problem. I > also can't find a good explanation as to what this parameter really does or > does not do even in the puppet server documentation. Anyone have a better > insight into this? > > Also is there a way to just turn off puppetdb? > > I've attached a file that is a snapshot of the puppetdb dashboard. > > Anyone experience anything like this? > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/27a1ac7d-d024-4489-bac8-77f41080d80f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
