Hi James, thanks for your thorough answer. We definitely have in mind that we should upgrade Puppet to 4.X to get all the benefits of Puppet Server, but until that happens we were wondering whether there was a simpler way to boost performance on the Puppet master until we can perform an upgrade to Puppet 4.x
Thanks, Cesar. On Thursday, 20 April 2017 14:59:00 UTC+2, James Pryor wrote: > > Cesar, > For general catalog compilation time improvements, you might be best > served by using Puppet Server ( > https://docs.puppet.com/puppetserver/1.2/services_master_puppetserver.html > ) instead of passenger. > > In 2014, my team & I performed our own sort of benchmark tests of the > puppet master server. From > https://indico.cern.ch/event/320819/contributions/742968/ download the > PDF > > https://indico.cern.ch/event/320819/contributions/742968/attachments/619020/851687/Configuration_Management_Change_Management-James-Pryor.pdf > and please see pages 32 - 37 that show the load on the single puppet > master that was running Apache + passenger. > > Approximately six months later 2015, another test was done on the same > hardware with the same software, but with the replacement of Apache & > Passenger with Puppet Server 1.x. From > https://indico.cern.ch/event/346931/contributions/817788/ download the > PDF > https://indico.cern.ch/event/346931/contributions/817788/attachments/684633/940417/Configuration_Management_at_the_RACF.pdf > and check pages 3 - 8 and see that compilation times were effectively cut > in half. > > So in summary, in an apples-to-apples comparison on the same hardware, > Puppet Server 1.x is close to twice as fast as Apache & Passenger. > Now there are some differences between Puppet Server & Passenger which is > documented here: > https://docs.puppet.com/puppetserver/1.2/puppetserver_vs_passenger.html > > Now all that said, Puppet 3.x has been End of Life-ed as of 2017-01-01 ( > https://groups.google.com/forum/#!topic/puppet-announce/tSe_4KTS7t0 ). > You should consider working toward a migration path from Puppet 3.3 to the > latest Puppet 3.8.x , and then look toward testing with Puppet 4.x, and > either land on Puppet 4.x or maybe even wait for Puppet 5.x (which is > hinted at coming in 2017 > https://groups.google.com/d/msg/puppet-dev/-H1pHJM6NLE/II3X8Cw6AwAJ). > > https://docs.puppet.com/puppet/4.10/index.html#upgrading-from-puppet-3 > https://docs.puppet.com/puppet/4.10/upgrade_major_pre.html > > Regards, > James > > > On Thu, Apr 20, 2017 at 6:32 AM, Cesar <cesar...@ammeon.com <javascript:>> > wrote: > >> Hi all, >> >> we have some performance issues with Puppet 3.3 catalog compilation (we >> are using Passenger to host Puppet master processes). >> It looks like a single Puppet master process is being used for the >> catalog compilation, and importing the catalogs is taking ~3 minutes. >> >> Apr 19 10:21:14 ieatlms3841 puppet-master[6700]: Handling request: GET >> /production/node/ieatrcxb3742 >> Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: Using cached facts for >> ieatrcxb3742 >> Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: Caching node for >> ieatrcxb3742 >> Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: node supports formats: >> pson b64_zlib_yaml yaml raw >> Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] >> 1.1 Rendered result in Puppet::Network::Format[pson]: took 0.0014 seconds >> Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] >> 1.2 Sent response: took 0.0000 seconds >> Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] >> 1 Processed request GET /production/node/ieatrcxb3742: took 0.0159 seconds >> ... >> >> Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.1 Setup server facts for compiling: took 0.0172 seconds >> Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: 'replace facts' command >> for ieatrcxb3742 submitted to PuppetDB with UUID >> 9a23c445-5203-4987-9faf-a187ac7dd0b3 >> Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.2 Found facts: took 0.1696 seconds >> Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: Using cached facts for >> ieatrcxb3742 >> Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: Caching node for >> ieatrcxb3742 >> Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.3 Found node information: took 0.0133 seconds >> *Apr 19 10:21:20* ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatlms3841.pp' in environment production >> Apr 19 10:21:27 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb3739.pp' in environment production >> Apr 19 10:21:42 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb3740.pp' in environment production >> Apr 19 10:21:51 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb3742.pp' in environment production >> Apr 19 10:21:56 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb3743.pp' in environment production >> Apr 19 10:22:05 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb3744.pp' in environment production >> Apr 19 10:22:11 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb3755.pp' in environment production >> ... >> >> Apr 19 10:24:03 ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb5894.pp' in environment production >> *Apr 19 10:24:06 *ieatlms3841 puppet-master[6700]: importing >> '/etc/puppet/manifests/plugins/ieatrcxb6430.pp' in environment production >> Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.4.1 Compile: Set node parameters: took 0.0025 seconds >> Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.4.2 Compile: Created settings scope: took 0.0295 seconds >> Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.4.3 Compile: Evaluated main: took 0.0083 seconds >> >> ... >> >> Apr 19 10:24:29 ieatlms3841 puppet-master[6700]: (Firewall[1018 >> vmmonitord ipv6]) Adding default for require >> Apr 19 10:24:29 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.4.7 Compile: Finished catalog: took 0.1885 seconds >> Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] >> 1.4 Compiled catalog for ieatrcxb3742 in environment production: took >> *191.6076* seconds >> Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: Compiled catalog for >> ieatrcxb3742 in environment production in 191.61 seconds >> Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: Caching catalog for >> ieatrcxb3742 >> >> >> I was wondering whether there is any way to reduce catalog compilation? >> Could the catalog compilation be split across different Puppet master >> processes or something like that? >> >> >> Thanks, >> Cesar. >> >> This email and any files transmitted with it are confidential and >> intended solely for the use of the individual or entity to whom they are >> addressed. If you have received this email in error please notify the >> system manager. This message contains confidential information and is >> intended only for the individual named. If you are not the named addressee >> you should not disseminate, distribute or copy this e-mail. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-users...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/c04da5d1-bb08-46a1-ab95-ffe3e52069a2%40googlegroups.com >> >> <https://groups.google.com/d/msgid/puppet-users/c04da5d1-bb08-46a1-ab95-ffe3e52069a2%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/691f5bae-e980-4998-a385-f4137840adbe%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
