hello,
----- Original Message -----
> From: "Paul Colby" <pco...@gmail.com>
> To: puppet-users@googlegroups.com
> Sent: Monday, August 13, 2012 6:22:10 PM
> Subject: [Puppet Users] hiera command line performance
>
> Hi everyone,
>
>
> The hiera command line program (version 0.3.0) is currently taking
> around 2 seconds per invocation. This seems quite slow to me, but is
> probably not an issue for most users.
2 seconds seem quite slow though i guess it depends on the size of your
yamls and so forth, sounds like yours are big and complex. I'd hope the
bulk of the time is spent on starting/stopping ruby each time.
A better option might be to consider writing your script in ruby rather
than bash so you can then do your lookup that way reusing the single
hiera instance and avoid the ruby start/stop cost
So a simple test on my setup doing:
for i in {1..100}
do
hiera syslocation ::location=foo
done
this takes 30 seconds, ages. Here is the same in Ruby reusing the
class and using a single interpreter:
h = Hiera.new(:config => "/etc/hiera.yaml")
Benchmark.measure do
100.times do
puts h.lookup("syslocation", "", {"::location" => "foo"})
end
end
this takes 0.1 second for 100 lookups. If you know a bit of ruby this
would be a good approach for you even if you just write your script to
configure a single machine in ruby and run it 40 times only
>
>
> We have a number of legacy machines that for one reason or another
> have not been puppetised yet, and probably won't be anytime soon, so
> I have a somewhat clever script that, for each of these hosts, uses
> the hiera command like utility to export a few puppet-templated
> documents just as puppet+hiera does for our puppeted server. This
> works very nicely, but involves ~250 hiera lookups per host, which
> at 2 seconds per lookup, is taking more than 8 minutes per host (and
> using near 100% of a CPU core the entire time). As we have ~40 such
> hosts, that's over 5 hours with no concurrency (we do use some
> concurrency, but since each invocation uses near 100% of a CPU core
> each, there's no point running more simultaneous hiera processes
> than the number of CPUs, and even then, we need some reserved for
> other services).
>
>
> So, is there some way I can make this export run faster? The bulk of
> the time seems to be spent in initialising the hiera command line
> program, not the actual data lookup (for example, if I fail to
> provide the necessary identity / scope files, it still takes 2
> seconds to get around to returning an error, but if I fail to
> provide a config file, it returns instantaneously with error). So,
> I'd either like to make the program load / initialise much faster,
> or be able to query multiple values for a single hiera command line
> invocation.
>
>
> Also, is there a more recent version of hiera that would be faster?
> (I'm a little unclear about hiera versioning - we're running 0.3.0,
> but I don't know if that's recent or old).
>
>
> Any suggestions / tips would be greatly appreciated :)
>
>
> Thanks.
>
>
> Paul.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/YTZZ7FOshJkJ .
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
