On Fri, Nov 30, 2018 at 7:39 AM Darragh Bailey <daragh.bai...@gmail.com>
wrote:
>
>
> On Thursday, November 29, 2018 at 11:27:13 AM UTC, Darragh Bailey wrote:
>>
>>
>>
>> On Wednesday, November 28, 2018 at 6:32:44 PM UTC, Eric Sorenson wrote:
>>>
>>> Hi Darragh, the fact that the error message contains a '400' error
>>> suggests the problem happens on the server when it receives the report.
>>>
>>> My first guess given that error message is also that there's a mix of
>>> versions installed, but it's weird that it only happens on some reports.
>>> Maybe there is something malformed in those reports that triggers a
>>> different code path on the server.
>>>
>>
>> Indeed there is, I missed that there were errors further back in the
>> output log previously and assumed the failure to post the report
>> successfully was the error. Seems they are not compilation errors but
>> rather runtime errors where something unexpected occurred in applying the
>> catalog.
>>
>> For successful runs there is no problem in posting back the report, this
>> at least now gives me something to reproduce consistently, just add an
>> error somewhere that will trigger it.
>>
>
> So confirmed I can reproduce this by simply adding the following block to
> the site.pp for our vagrant environment
>
> exec { 'break the run':
> command => '/bin/false',
> }
>
> However it only triggers the error on the first run, once there is a
> successful run, can't trigger the problem subsequently.
>
This makes me think it's related to the transactionstore functionality (it
keeps track of what the "current" value is the last time puppet applied a
catalog, so that it can tell if a change is due to an intentional manifest
change or the system drifted and needed to be remediated). I say that
because the transaction store records information differently depending if
the event was successful or not. Might want to look in `puppet config print
transactionstorefile --section agent` on the agent to see if there are any
clues.
Also earlier you mentioned:
> Warning: Event['previous_value'] contains a Process::Status value. It
will be converted to the String 'pid 30408 exit 1'
The "previous_value" for an exec should always be "notrun" or the desired
value (due to the onlyif/unless/creates checks causing the exec to skip). I
wonder if somehow the previous system from the transactionstore is ending
up in the next report?
>>
>>> You can save a copy of the reports by adding `store` to the type of
>>> report submission on the master: `reports = https,store` and see what they
>>> look like. They should go into a subdirectory
>>> of /opt/puppetlabs/puppet/cache/reports
>>>
>>
>>> HTH
>>> --eric0
>>>
>>
>> Ah, fantastic, didn't think of that! That will definitely help, many
>> thanks.
>>
>>
> So apparently 'store' is the default based on
> https://puppet.com/docs/puppet/5.4/reporting_about.html#configuring-reporting
> and it should be 'http' just in case anyone else encounters this.
>
> But it doesn't appear to have any effect in capturing the report being
> sent. Might need to make use of nginx and report_port as a way to forward
> on the request while logging it's content it so that I can grab what was
> sent.
>
> Looking on the agent node I see the
> /var/lib/puppet/state/last_run_summary.yaml containing
>
> events:
> failure: 1
> success: 362
> total: 363
>
> But unfortunately there's no /var/lib/puppet/state/last_run_report.yaml
> that corresponds, as it doesn't appear to get updated due to the failure,
> so there's no actual record of the report being sent to the master.
>
You may be running into https://tickets.puppetlabs.com/browse/PUP-6708?
> --
> Darragh Bailey
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/45e5e998-f3a1-4aa1-8066-5bed2a4dab29%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/45e5e998-f3a1-4aa1-8066-5bed2a4dab29%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
Josh Cooper | Software Engineer
j...@puppet.com | @coopjn
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CA%2Bu97ukEMbCtZZzZEP5L1Y1Gjjwy1BwMuUCGrwOcU3Cpu49TcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
