"Redo SSL setup after changing certificates If you’ve recently changed the certificates in use by the PuppetDB server, you’ll also need to update the SSL configuration for PuppetDB itself.
If you’ve installed PuppetDB from Puppet packages, you can simply re-run the puppetdb ssl-setup command. Otherwise, you’ll need to again perform the SSL configuration steps outlined in the installation instructions <https://puppet.com/docs/puppetdb/latest/install_from_source.html>." https://puppet.com/docs/puppetdb/latest/maintain_and_tune.html On Monday, April 27, 2020 at 5:17:39 PM UTC+10, Yvan Broccard wrote: > > Hi, > > I'm struggling with a simple update of PuppetDB since a couple of days, > without finding the problem. > I have 4 PuppetServers running Puppetserver 6.9 > (puppetserver-6.9.0-1.el7.noarch). One has the CA role, the 3 others are > simple masters. I have one dedicated PuppetDB server > running puppetdb-6.5.0-1. > > Everything is working like a charm since a couple of years. It was updated > from Puppet 3, 4 and 6 without a glitch. Everything is running on CentOS 7. > > Now, when I want to update PuppetDb from 6.5 to 6.9, nothing works anymore. > > All nodes are complaining with these messages : > > Warning: Unable to fetch my node definition, but the agent run will > continue: > Warning: Error 500 on SERVER: Server Error: Could not retrieve facts for > vmlabybr06.staging.rsvgnw.local: Failed to find facts from PuppetDB at > vmprdpuppet41.rsvgnw.local:8140: Failed to execute > '/pdb/query/v4/nodes/vmlabybr06.staging.rsvgnw.local/facts' on at least 1 > of the following 'server_urls': https://vmctldeploy20.rsvgnw.local:8081 > Info: Retrieving pluginfacts > Info: Retrieving plugin > Info: Retrieving locales > Info: Loading facts > Error: Could not retrieve catalog from remote server: Error 500 on SERVER: > Server Error: Failed to execute > '/pdb/cmd/v1?checksum=5da252cdae0fc1737726e9ace846d74856395703&version=5&certname=vmlabybr06.staging.rsvgnw.local&command=replace_facts&producer-timestamp=2020-04-09T13:15:44.382Z' > > on at least 1 of the following 'server_urls': > https://vmctldeploy20.rsvgnw.local:8081 > Warning: Not using cache on failed catalog > Error: Could not retrieve catalog; skipping run > > > In the server log I get this : > > 2020-04-09T15:22:45.169+02:00 WARN [qtp1002336767-143] > [c.p.h.c.i.PersistentSyncHttpClient] Error executing http request > javax.net.ssl.SSLException: Received fatal alert: handshake_failure > at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647) > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615) > at > sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781) > at > sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070) > at > sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) > at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) > at > org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:271) > at > org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316) > at > org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:503) > at > org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120) > at > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) > at > org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) > at > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) > at java.lang.Thread.run(Thread.java:748) > 2020-04-09T15:22:45.171+02:00 WARN [qtp1002336767-143] [puppetserver] > Puppet Error connecting to vmctldeploy20.rsvgnw.local on 8081 at route > /pdb/cmd/v1?checksum=0f8f2f1e474b2f551f6dc656bff34f1e43e56f6b&version=8&certname=vmlabvmt01.rsvgnw.local&command=store_report&producer-timestamp=2020-04-09T13:22:45.130Z, > > error message received was 'Error executing http request'. Failing over to > the next PuppetDB server_url in the 'server_urls' list > 2020-04-09T15:22:45.172+02:00 ERROR [qtp1002336767-143] [puppetserver] > Puppet Failed to execute > '/pdb/cmd/v1?checksum=0f8f2f1e474b2f551f6dc656bff34f1e43e56f6b&version=8&certname=vmlabvmt01.rsvgnw.local&command=store_report&producer-timestamp=2020-04-09T13:22:45.130Z' > > on at least 1 of the following 'server_urls': > https://vmctldeploy20.rsvgnw.local:8081 > > > I have checked a few things : > - Updated puppetdb-termini on the puppet-master from 6.5 to 6.9 (no change) > - added "verify_client_certificate = false" > to /etc/puppetlabs/puppet/puppetdb.conf on the masters (no change) > - added full certs list to PuppetDB > server /etc/puppetlabs/puppetdb/ssl/public.pem > > I've read there has been a change liked to SSL in the PuppetDB 6.6 > CHANGELOG. > > Here is what happens when I try to connect with openssl for > troubleshooting, to PuppetDB 6.5 > > openssl s_client -host puppetdb -port 8081 -CAfile > /etc/puppetlabs/puppet/ssl/certs/ca.pem > CONNECTED(00000003) > Can't use SSL_get_servername > depth=1 CN = Puppet CA: vmctldeploy10.rsvgnw.local > verify return:1 > depth=0 CN = vmctldeploy20.rsvgnw.local > verify return:1 > 140503727654720:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert > bad certificate:ssl/record/rec_layer_s3.c:1544:SSL alert number 42 > --- > Certificate chain > 0 s:CN = vmctldeploy20.rsvgnw.local > i:CN = Puppet CA: vmctldeploy10.rsvgnw.local > --- > Server certificate > -----BEGIN CERTIFICATE----- > ... > -----END CERTIFICATE----- > subject=CN = vmctldeploy20.rsvgnw.local > > issuer=CN = Puppet CA: vmctldeploy10.rsvgnw.local > > --- > Acceptable client certificate CA names > CN = Puppet CA: vmctldeploy10.rsvgnw.local > Client Certificate Types: RSA sign, DSA sign, ECDSA sign > Requested Signature Algorithms: > ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1 > Shared Requested Signature Algorithms: > ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1 > Peer signing digest: SHA256 > Peer signature type: RSA > Server Temp Key: ECDH, P-256, 256 bits > --- > SSL handshake has read 2213 bytes and written 455 bytes > Verification: OK > --- > > > The only way to go back is doing a full "revert to snaphot", as the db is > migrated between 6.5 and 6.9 > > Any advise welcome ! > > Cheers > Yvan > > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7d3541aa-c71c-41b9-b0cc-3c2885d26108%40googlegroups.com.
