On Tuesday, May 21, 2013 8:18:58 AM UTC-5, phundisk wrote: > > Occasionally I am seeing this error below on puppet client runs from > several different clients. The clients and the server in this environment > are rather old and this started to happen more after I used to nagios > module which uses stored configs. Does anyone know where this slowness > might be coming from (client, server, db, etc) or how I can increase this > timeout parameter? Or does this indicate some poor programming in my > puppet code in terms of inefficiency. I would say this happens maybe one > or two times a week out of 20 servers running every 30 minutes. > > *Current Setup:* > Puppet Server: puppet-server-2.7.20-1 > Ruby: 1.8.7.370 > Stored Configs: MySQL 5.5.15 > Cron Run Command (no service): /usr/sbin/puppetd --server puppet --onetime > --no-daemonize --verbose --logdest syslog > > *Error* > /usr/lib/ruby/1.8/timeout.rb:54:in `rbuf_fill': execution expired > (Timeout::Error) > from /usr/lib/ruby/1.8/timeout.rb:56:in `timeout' > from /usr/lib/ruby/1.8/timeout.rb:76:in `timeout' > from /usr/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill' > from /usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil' > from /usr/lib/ruby/1.8/net/protocol.rb:126:in `readline' > from /usr/lib/ruby/1.8/net/http.rb:2020:in `read_status_line' > from /usr/lib/ruby/1.8/net/http.rb:2009:in `read_new' > from /usr/lib/ruby/1.8/net/http.rb:1050:in `request' > ... 24 levels... > from /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in > `run' > from /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in > `exit_on_fail' > from /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in > `run' > from /usr/sbin/puppetd:159 > >
It looks like you are running about at the edge of your master's capacity. Especially since you're running out of cron, that could be related to an uneven workload, where all of your servers make catalog requests at approximately the same time. If you have not already done so, then you should ensure that your crontab entries spread out requests relatively evenly over time. Are you running the master via its built-in webrick server, or have you set it up in apache/passenger, mongrel, or some other more featureful rack server? The webrick version is single-threaded, serving only one client at a time, and that can cause delays and client-side timeouts when multiple requests arrive close together, even if the server has enough raw power that it should be able to serve them all adequately. Twenty clients is in the range where the single-threaded webrick server may start to have trouble keeping up, though that depends heavily on your manifests and the pattern of requests. I have a webrick server supporting many more clients than that, but the manifest set it's working with is pretty simple. Nagios resources themselves are not a special problem, but exported resources and stored configs very well could be. Do use thin stored configs if possible, as that will lighten the load somewhat. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
