On Wed, Jul 6, 2011 at 1:33 PM, Derek <dtam...@gmail.com> wrote: > I am working on building a facter tag based node classifier similar to > > https://github.com/jordansissel/puppet-examples/tree/master/nodeless-puppet/ > . > However, I have run into an issue where I cannot use puppet's require > file ability to push the yaml file containing the facts file to the > client because it would require two runs of puppet to pickup changes. > Consequently, I have written into the facter ruby script the ability > to connect to puppet's restful api and get the yaml file from the > private store. This works fine in irb, ruby, and facter if called > directly. However, when run inside of a puppet run it seems to fail on > parsing the http response correctly into yaml. As a result, it does > not get saved to disk and loaded as a fact for the puppet run. > > There is probably a simpler way to do this. Essentially we want to > have tags on a server and use that to selectively include or remove > modules from a server by facter tags rather than by a server's name. > > Some Version Information: > - os = CentOS release 5.2 (Final) > - ruby = ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux] > - facter = 1.6.0 (updated because my script loads multiple facts and > the older version we were running requires the filename to match the > fact name. This was not working because I did not want to split my > ruby load script into multiple files to match each of the fact names.) > - puppet = 0.25.4 > > Yaml file it is trying to grab from a private store: > --- > role: > - base > - db > env: > - dev > > The yaml file downloads correctly via a puppet run without my script. > I can also wget the file and use net/https via ruby to get the file. > All methods return the correct file with matching md5sums. > > Under my module called "truth" I have the following: > - files -> private -> domain.inter -> hostname -> truth_tags.yml > ex: > --- > role: > - base > env: > - dev > > - lib -> facter -> load_truth_tags.rb > problem area: > def apitruthtag(calltype) > > # set some client side variables to build on later > sslbasedir = '/etc/puppet/ssl' > sslprivdir = sslbasedir + '/private_keys' > sslpubdir = sslbasedir + '/certs' > sslcafile = sslpubdir + '/ca.pem' > > # this sets if we want metadata or content from puppet > datatype = calltype > > # We want yaml back from puppet > header = {'Accept' => 'yaml'} > > # Setup some connection variables to our puppet server and what we > want from it > proto = 'https' > server = 'puppet.domain.inter' > port = '8140' > path = '/production/file_' + datatype + '/truth_private/ > truth_tags.yml' > > # Build the full uri to request from our puppet server. Then parse > it for port and things > uri = URI.parse(proto + '://' + server + ':' + port + path) > > # Setup the http module and set it for getting data > http = Net::HTTP.new(uri.host, uri.port) > request = Net::HTTP::Get.new(uri.request_uri, header) > > http.use_ssl = true if uri.scheme == 'https' > > # Enable ssl verification to ensure we are talking to the correct > people > http.verify_mode = OpenSSL::SSL::VERIFY_PEER > > # Cert Auth: > # Set certificate paths > # puppet certificate authority file > > if File.readable?(sslcafile) then > # Puppet ca file > http.ca_file = sslcafile > puts "readable? " + sslprivdir + '/' + hostname + '.pem' if $debug > if File.readable?(sslprivdir + '/' + hostname + '.pem') then > # client private key > http.key = OpenSSL::PKey::RSA.new(File.read(sslprivdir + '/' + > hostname + '.pem')) > puts "readable? " + sslpubdir + '/' + hostname + '.pem' if > $debug > if File.readable?(sslpubdir + '/' + hostname + '.pem') then > # client public key > http.cert = OpenSSL::X509::Certificate.new(File.read(sslpubdir > + '/' + hostname + '.pem')) > > # Make the request > response = http.request(request) > else > raise "No readable client pubic key in #{sslpubdir}/ > #{hostname}.pem" > end # End public key check > else > raise "No readable client private key in #{sslprivdir}/ > #{hostname}.pem" > end # End private key check > else > raise "No readable ca cert in #{sslcafile}" > end # End ca file check > > # Check to make sure we got some data back > if response != nil > # Check to see if we have a good server response before saving the > variable > puts "check code " + response.code if $debug > if ((response.code < "300") and (response.code >= "200")) > return response.body > else > raise "server did not return an acceptable reponse code" > end # end server response code check > else > raise "No response from #{server}" > end # end nil response check > > end # end apitruthtag > > servermd5 = YAML.load(apitruthtag("metadata")).ivars["checksum"] # > When executed from a puppet run I tells me that ivars is undefined. > > - lib -> puppet -> parser -> functions -> truth_tags.rb > - manifests -> init.pp > ex: > class truth inherits truth::init_bootstrap { > if truth_tag('role', 'base') and !truth_tag('role', 'nobase') { > notice("${::hostname}: Including role, base modules...") > > notice(" ${::hostname}: role, base: including network") > include network > } > } > - manifests -> init_bootstrap.pp > just makes sure the /etc/truth_tags.yml file exists on disk > > Process (if the facter yaml load was working): > 1. puppet client downloads "lib -> facter -> load_truth_tags.rb". > 2. facter runs with the external ruby fact script. > - this fails on puppet runs but functions correctly directly in > ruby, irb, and facter itself using "facter -d" > - If working load_truth_tags.rb would do this: > 1. See if a cached /etc/truth_tags.yml file exists > 2. if it does exist then it md5 hashes the file. Next, it uses net/ > https to connect to the resful api to get the puppetmaster's hash to > see if the local file has changed. If they differ it pulls down the > changed file from puppetmaster and writes it to disk. > 3. if the file does not exist on disk it pulls it down from > puppetmaster using net/https and checks both the server and client > hashes to see if it was modified in transit and that it downloaded the > correct content. > 3. Next, manifests -> init.pp loads the truth_tags function from lib - > > puppet -> parser -> functions -> truth_tags.rb for the puppet run. > - the manifest init.pp file is basically a set of rules that says if > role = db then include these other functions. The goal being we don't > want to classify servers by name but rather by their functions. This > would allow us to be more flexible and not have to worry about double > including things in the nodes.pp file. > > > > Sorry I have not included all of the "load_truth_tags.rb" script as it > is long (317 lines). I can if requested. > > Issue I can see which differs from a puppet run vs stand alone ruby > run: > > irb, ruby, or facter: > yaml parsed http response = #<YAML::Object:0x2ada01f7cf00> > > puppet run: > yaml parsed http response = #<Puppet::FileServing::Metadata: > 0x2ac7987b9c08> > with error: > > undefined method `ivars' for #<Puppet::FileServing::Metadata: > 0x2ac7987152c0> > #<NoMethodError: undefined method `ivars' for > #<Puppet::FileServing::Metadata:0x2ac7987152c0>> > > and traceback: > > /var/puppet/lib/facter/load_truth_tags.rb:253:in `calcservertruthmd5' > /var/puppet/lib/facter/load_truth_tags.rb:268 > /opt/ruby-1.8.6-p287/lib/ruby/1.8/timeout.rb:62:in `timeout' > /var/puppet/lib/facter/load_truth_tags.rb:37 > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 73:in `load' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 73:in `load_file' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 38:in `load_all' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 33:in `each' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 33:in `load_all' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 30:in `each' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/loader.rb: > 30:in `load_all' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter/util/collection.rb: > 94:in `load_all' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/facter.rb:218:in > `loadfacts' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/configurer/ > fact_handler.rb:61:in `reload_facter' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/configurer/ > fact_handler.rb:18:in `find_facts' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/configurer/ > fact_handler.rb:29:in `facts_for_uploading' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/configurer.rb: > 100:in `retrieve_catalog' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/configurer.rb: > 162:in `run' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/agent.rb:53:in > `run' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/agent/locker.rb: > 21:in `lock' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/agent.rb:53:in > `run' > /opt/ruby-1.8.6-p287/lib/ruby/1.8/sync.rb:229:in `synchronize' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/agent.rb:53:in > `run' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/agent.rb:134:in > `with_client' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/agent.rb:51:in > `run' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/application/ > puppetd.rb:103:in `onetime' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/application.rb: > 226:in `send' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/application.rb: > 226:in `run_command' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/application.rb: > 217:in `run' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/application.rb: > 306:in `exit_on_fail' > /opt/ruby-1.8.6-p287/lib/ruby/site_ruby/1.8/puppet/application.rb: > 217:in `run' > /usr/sbin/puppetd:159 > > > > > > > > > It appears that yaml is not parsing correctly when run inside a puppet > run. As a result, the response.body.ivars["checksum"] variable is not > being set. Any ideas? > > So this kind of rare, but you've almost provided too much info here :) and I'm finding it difficult to work out what your actual problem is.
To back up a little... "There is probably a simpler way to do this. Essentially we want to have tags on a server and use that to selectively include or remove modules from a server by facter tags rather than by a server's name." If you're looking everything up by hostname anyway, why not just have an external node classifier that does all this logic server-side? -- Nigel Kersten Product Manager, Puppet Labs Twitter: @nigelkersten *Join us for **PuppetConf *<http://www.bit.ly/puppetconfsig> September 22nd and 23rd in Portland, Oregon, USA. * * -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
