Cesar, For general catalog compilation time improvements, you might be best served by using Puppet Server ( https://docs.puppet.com/puppetserver/1.2/services_ master_puppetserver.html ) instead of passenger.
In 2014, my team & I performed our own sort of benchmark tests of the puppet master server. From https://indico.cern.ch/event/ 320819/contributions/742968/ download the PDF https://indico.cern.ch/event/320819/contributions/742968/ attachments/619020/851687/Configuration_Management_ Change_Management-James-Pryor.pdf and please see pages 32 - 37 that show the load on the single puppet master that was running Apache + passenger. Approximately six months later 2015, another test was done on the same hardware with the same software, but with the replacement of Apache & Passenger with Puppet Server 1.x. From https://indico.cern.ch/event/ 346931/contributions/817788/ download the PDF https://indico.cern.ch/event/ 346931/contributions/817788/attachments/684633/940417/ Configuration_Management_at_the_RACF.pdf and check pages 3 - 8 and see that compilation times were effectively cut in half. So in summary, in an apples-to-apples comparison on the same hardware, Puppet Server 1.x is close to twice as fast as Apache & Passenger. Now there are some differences between Puppet Server & Passenger which is documented here: https://docs.puppet.com/puppetserver/1.2/puppetserver_vs_passenger.html Now all that said, Puppet 3.x has been End of Life-ed as of 2017-01-01 ( https://groups.google.com/forum/#!topic/puppet-announce/tSe_4KTS7t0 ). You should consider working toward a migration path from Puppet 3.3 to the latest Puppet 3.8.x , and then look toward testing with Puppet 4.x, and either land on Puppet 4.x or maybe even wait for Puppet 5.x (which is hinted at coming in 2017 https://groups.google.com/d/ msg/puppet-dev/-H1pHJM6NLE/II3X8Cw6AwAJ). https://docs.puppet.com/puppet/4.10/index.html#upgrading-from-puppet-3 https://docs.puppet.com/puppet/4.10/upgrade_major_pre.html Regards, James On Thu, Apr 20, 2017 at 6:32 AM, Cesar <cesar.mu...@ammeon.com> wrote: > Hi all, > > we have some performance issues with Puppet 3.3 catalog compilation (we > are using Passenger to host Puppet master processes). > It looks like a single Puppet master process is being used for the catalog > compilation, and importing the catalogs is taking ~3 minutes. > > Apr 19 10:21:14 ieatlms3841 puppet-master[6700]: Handling request: GET > /production/node/ieatrcxb3742 > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: Using cached facts for > ieatrcxb3742 > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: Caching node for > ieatrcxb3742 > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: node supports formats: > pson b64_zlib_yaml yaml raw > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] > 1.1 Rendered result in Puppet::Network::Format[pson]: took 0.0014 seconds > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] > 1.2 Sent response: took 0.0000 seconds > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] > 1 Processed request GET /production/node/ieatrcxb3742: took 0.0159 seconds > ... > > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.1 Setup server facts for compiling: took 0.0172 seconds > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: 'replace facts' command > for ieatrcxb3742 submitted to PuppetDB with UUID > 9a23c445-5203-4987-9faf-a187ac7dd0b3 > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.2 Found facts: took 0.1696 seconds > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: Using cached facts for > ieatrcxb3742 > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: Caching node for > ieatrcxb3742 > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.3 Found node information: took 0.0133 seconds > *Apr 19 10:21:20* ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatlms3841.pp' in environment production > Apr 19 10:21:27 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3739.pp' in environment production > Apr 19 10:21:42 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3740.pp' in environment production > Apr 19 10:21:51 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3742.pp' in environment production > Apr 19 10:21:56 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3743.pp' in environment production > Apr 19 10:22:05 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3744.pp' in environment production > Apr 19 10:22:11 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3755.pp' in environment production > ... > > Apr 19 10:24:03 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb5894.pp' in environment production > *Apr 19 10:24:06 *ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb6430.pp' in environment production > Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.1 Compile: Set node parameters: took 0.0025 seconds > Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.2 Compile: Created settings scope: took 0.0295 seconds > Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.3 Compile: Evaluated main: took 0.0083 seconds > > ... > > Apr 19 10:24:29 ieatlms3841 puppet-master[6700]: (Firewall[1018 vmmonitord > ipv6]) Adding default for require > Apr 19 10:24:29 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.7 Compile: Finished catalog: took 0.1885 seconds > Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4 Compiled catalog for ieatrcxb3742 in environment production: took > *191.6076* seconds > Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: Compiled catalog for > ieatrcxb3742 in environment production in 191.61 seconds > Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: Caching catalog for > ieatrcxb3742 > > > I was wondering whether there is any way to reduce catalog compilation? > Could the catalog compilation be split across different Puppet master > processes or something like that? > > > Thanks, > Cesar. > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you have received this email in error please notify the system manager. > This message contains confidential information and is intended only for the > individual named. If you are not the named addressee you should not > disseminate, distribute or copy this e-mail. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/puppet-users/c04da5d1-bb08-46a1-ab95-ffe3e52069a2%40googlegroups.com > <https://groups.google.com/d/msgid/puppet-users/c04da5d1-bb08-46a1-ab95-ffe3e52069a2%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAOsPUcit6_%3DvQPunUb%2BZpdqAG-yjEGaoVHya1HZVQgHvWff51w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
