Hey Trevor,
I've been able to reproduce your problem and I see what's going on. Since
this sort of post hoc catalog serialization isn't something that's done a
lot (or at all) there are some sharp edges; I'm going to head into this
code base with a big stick and I'm going to hit things really hard until
they work. It might take me a few days but when I've got things a bit more
sorted out I'll let you know what I've figured out.
Adrien
On Thu, Mar 24, 2016 at 6:48 AM Trevor Vaughan <tvaug...@onyxpoint.com>
wrote:
> Hey Adrian,
>
> The code is pretty simple right now and is as follows:
>
> ### Type
>
> module Puppet
> newtype(:catalog_test) do
> @doc = "Save a micro-catalog"
>
> newparam(:name) do
> isnamevar
> desc "Resource name to trigger off of"
> end
>
> newproperty(:resource_type) do
> desc "The resource type"
> end
> end
> end
>
> ### Provider
>
> Puppet::Type.type(:catalog_test).provide(:micro_catalog) do
> desc 'Save a micro-catalog'
>
> def resource_type
> my_catalog = Puppet::Resource::Catalog.new
>
> target_resource = @resource.catalog.resources.find{|x|
> (x.type == @resource[:resource_type].downcase.to_sym) &&
> (x.name == @resource[:name])
> }
>
> if target_resource
>
> @resource.catalog.relationship_graph.dependencies(target_resource).each do
> |res|
> my_catalog.add_resource(res)
> end
>
> my_catalog.add_resource(@resource)
>
>
> @resource.catalog.relationship_graph.dependents(target_resource).each do
> |res|
> my_catalog.add_resource(res)
> end
> end
>
> # Tinker Point
> require 'pry'
> binding.pry
> return @resource[:resource_type]
> end
> end
>
> ### Test File
>
> file { '/tmp' : ensure => 'directory' }
>
> file { '/tmp/foo1': content => 'blah' }
>
> file { '/tmp/stack': ensure => 'directory' }
>
> file { '/tmp/stack/test1': content => 'test' }
>
> file { '/usr/test':
> content => 'test',
> require => File['/tmp/stack']
> }
>
> file { '/var/trigger':
> content => 'test',
> notify => File['/tmp/stack/test1']
> }
>
> catalog_test { '/var/trigger':
> resource_type => 'file'
> }
>
> Thanks for the help,
>
> Trevor
>
> On Wed, Mar 23, 2016 at 2:43 PM, Adrien Thebo <adr...@puppetlabs.com>
> wrote:
>
>> The exact issues that you may run into depend on how the catalog was
>> generated and what it's storing; in my scenario I'm using the RAL to
>> collect Puppet::Type instances, I'm converting them to Puppet::Resource
>> instances, and then inserting those into the catalog. Trying to
>> individually convert resources in the catalog might be messy because you'll
>> need to delete the type you're converting and then add the Resource
>> instance; it's doable but there are some logistics around that.
>>
>> Do you have code that you can provide to demonstrate what you're doing?
>> Given a RAL catalog you should be able to get a resource catalog in some
>> form, as for reactivating a catalog I'm not sure what you mean by
>> "reactivate". And lastly, the "resource catalog" is effectively the only
>> means of serializing a catalog right now; if you want to store a catalog
>> you'll almost certainly have to convert your data to a resource catalog, or
>> figure out how you can make Puppet::Type instances serializable.
>>
>> On Tue, Mar 22, 2016 at 8:43 AM Trevor Vaughan <tvaug...@onyxpoint.com>
>> wrote:
>>
>>> Hey Adrian,
>>>
>>> Thanks for the response. I tried the
>>> Puppet::Resource::Catalog#to_resource method but ended up with an error on
>>> 'copy_as_resource' not found.
>>>
>>> I also tried the 'to_resource' method on all of the resource in my
>>> catalog individually but that didn't seem to get me anywhere either.
>>>
>>> Looking at the links that you posted, I suppose what I'm trying to do is
>>> actually save a RAL catalog and then reactivate it via the command line at
>>> a later point. Do you know if this is possible? I don't really need a
>>> compiler-level resource catalog since this won't be shipped off of the
>>> local system.
>>>
>>> Thanks,
>>>
>>> Trevor
>>>
>>> On Mon, Mar 21, 2016 at 11:16 AM, Adrien Thebo <adr...@puppetlabs.com>
>>> wrote:
>>>
>>>> I've been working on a project with similar goals and ran into the same
>>>> thing. The core of the issue is that there are two catalogs, the resource
>>>> catalog and the RAL catalog. The resource catalog is what the Puppet
>>>> compiler emits and contains Puppet::Resource instances; this is what's
>>>> actually sent to the agent and what you'll see if you run `puppet master
>>>> --compile`. The RAL catalog is generated when the catalog is actually being
>>>> applied (
>>>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/configurer.rb#L106),
>>>> and converts Puppet::Resource instances to Puppet::Type instances.
>>>> Puppet::Resource instances are meant to be serialized, but Puppet::Type
>>>> instances aren't - I'm pretty sure that the error you're seeing is because
>>>> Puppet is trying to serialize Puppet::Type instances that don't have the
>>>> correct method defined.
>>>>
>>>> There is a document in the Puppet source (
>>>> https://github.com/puppetlabs/puppet/blob/master/docs/catalogs.md)
>>>> that contains a bit more information about the different types of catalogs.
>>>>
>>>> That being said there's a couple of ways of converting things around to
>>>> a more easily serializable format.
>>>>
>>>> One option is to individually convert Puppet::Type instances to
>>>> Puppet::Resource instances as you find them; I've been doing them with
>>>> something like this:
>>>>
>>>> Puppet::Type.type(:service).instances.map { |res| res.to_resource }
>>>>
>>>> Alternately, if you have an existing catalog and you want to convert
>>>> and serialize that wholesale, you should be able to use
>>>> `Puppet::Resource::Catalog#to_resource` (
>>>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/resource/catalog.rb#L490)
>>>> to do this. I haven't tested this out but it seems like it should work.
>>>>
>>>> There are a couple of caveats in this - for instance things get a
>>>> little bit screwy with some resource types such as files, but if you run
>>>> into issues with too many parameters being generated/not enough parameters
>>>> being generated, let me know.
>>>>
>>>> If you make progress in this area let me know; since I'm doing some
>>>> poking around in this area I would be interested in seeing what you find
>>>> out as well.
>>>>
>>>>
>>>>
>>>> On Sat, Mar 19, 2016 at 12:04 PM Trevor Vaughan <tvaug...@onyxpoint.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I'm doing some experiments with extracted catalog snippets but seem to
>>>>> be having issues on saving the new catalog after the fact.
>>>>>
>>>>> I have a Puppet::Resource::Catalog object that has content and
>>>>> relationships but when I attempt to save it using 'to_pson' I get an
>>>>> "undefined method `to_pson_data_hash`" for any given resource in the
>>>>> catalog.
>>>>>
>>>>> As an aside, if I just run @resource.catalog.to_pson from within a
>>>>> provider, I end up with the same error.
>>>>>
>>>>> Any help would be appreciated.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Trevor
>>>>>
>>>>> --
>>>>> Trevor Vaughan
>>>>> Vice President, Onyx Point, Inc
>>>>> (410) 541-6699
>>>>>
>>>>> -- This account not approved for unencrypted proprietary information --
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Puppet Developers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to puppet-dev+unsubscr...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUnc589eJ3nujpF0G38%3D6hAHOMHVShU7xRU9KfzS%2BTjQg%40mail.gmail.com
>>>>> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUnc589eJ3nujpF0G38%3D6hAHOMHVShU7xRU9KfzS%2BTjQg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>> --
>>>> Adrien Thebo | Puppet Labs
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Puppet Developers" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to puppet-dev+unsubscr...@googlegroups.com.
>>>>
>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/puppet-dev/CALVJ9SKOK4HyZyOMYT%3DF31m7GxdL0jVvmYvKhzUAPBb5qhn0wg%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SKOK4HyZyOMYT%3DF31m7GxdL0jVvmYvKhzUAPBb5qhn0wg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>
>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> --
>>> Trevor Vaughan
>>> Vice President, Onyx Point, Inc
>>> (410) 541-6699
>>>
>>> -- This account not approved for unencrypted proprietary information --
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Puppet Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to puppet-dev+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU7jp3SBnL2%3DOpa3abixQwPuUvWDgh6VEGRNntGjO8zsw%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU7jp3SBnL2%3DOpa3abixQwPuUvWDgh6VEGRNntGjO8zsw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> Adrien Thebo | Puppet Labs
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to puppet-dev+unsubscr...@googlegroups.com.
>>
> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJ6OjCnWG5TVqOMNFJF8Jc3kXvhGP9wgubg2CwET1Fy%3DA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJ6OjCnWG5TVqOMNFJF8Jc3kXvhGP9wgubg2CwET1Fy%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>
>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Trevor Vaughan
> Vice President, Onyx Point, Inc
> (410) 541-6699
>
> -- This account not approved for unencrypted proprietary information --
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWBPxK5fSfB5CGFZWyWc_PN7htatdChp4S%3DbZ9PayEiSg%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWBPxK5fSfB5CGFZWyWc_PN7htatdChp4S%3DbZ9PayEiSg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
Adrien Thebo | Puppet Labs
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJe2bkJT9W07kRO4YcOtPAm35BqMYFJVvWKxuVgQoq8xQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
