On 15/12/10 21:10, Disconnect wrote: > As a datapoint, this exact config (with mongrel_cluster) was working > great under 0.25.x. With fewer, slower cpus, slower storage (vm image > files) and 2G of ram...
So I ask it again: could it be a problem with passenger more than an issue with puppet itself? It would really be interesting to use some ruby introspection[1] to find exactly where the cpu is spent in those masters. Like with passenger it reparses everything instead of just compiling? (I simply don't know, just throwing out some ideas) I myself use nginx + mongrel, but have only a dozen of nodes, so I don't really qualify. > I gave puppet-load a try, but it is throwing errors that I don't have > time to dig into today: > debug: reading facts from: puppet.foo.com.yaml > /var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:72:in > `send_request': uninitialized constant EventMachine::ConnectionError > (NameError) > from > /var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:59:in > `setup_request' > from > /var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:49:in > `get' > from ./puppet-load.rb:272:in `spawn_request' > from ./puppet-load.rb:334:in `spawn' Could it be that you're missing EventMachine? > Running about 250 nodes, every 30 minutes. Did you try to use mongrel? Do you use splay time? Just some math (which might be totally wrong), to give an idea of how I think we can compute our optimal scaling case: So with 250 nodes and a sleep time of 30 minutes, we need to overcome 250 compiles in every 30 minute time spans. If we assume a concurrency of 2 and all nodes evenly spaced (in time), that means we must compile 125 nodes in 30 minutes. If each compilation takes about 10s, then that means it'll take 1250s, which means 20 minutes so you have some room for growth :) Now during those 20min your 2 master processes will consume 100% CPU. Since we're consuming the the CPU for only 66% of the 30 minute span, you'll globally consume 66% of all your CPU available... Hope that helps, [1]: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Introspection -- Brice Figureau My Blog: http://www.masterzen.fr/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
