Just to add, and I apologize for not noticing this before, but we do not use it for anything other than internal FreeIPA operations, but it appears the CA itself is not working. I cannot enroll new hosts because of this. Login to existing hosts seems to still work without issue however.
On Thu, Feb 6, 2025 at 12:45 PM Rob Crittenden via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote: > Russell and I did a bit of offline troubleshooting but unfortunately > didn't find anything. > > I'm cc'ing a couple of the PKI developers. They would know know how to > verify that the webapp(s) are registered properly and may be able to > tell us why we're seeing 404's. > > rob > > Russell Long wrote: > > With pki-tomcatd@pki-tomcat running, the ipa-cert-show gives a > > connection error: > > > > [root@ipa-primary ~]# openssl x509 -serial -noout -in /etc/ipa/ca.crt > > serial=01 > > [root@ipa-primary ~]# ipa cert-show 01 > > ipa: ERROR: Request failed with status 404: Non-2xx response from CA > > REST API: 404. (404) > > > > --Russ > > > > On Wed, Feb 5, 2025 at 2:06 PM Rob Crittenden <rcrit...@redhat.com > > <mailto:rcrit...@redhat.com>> wrote: > > > > Russell Long wrote: > > > Here is the obfuscated sosreport. > > > > It looks like the CA restart that happened immediately before the > first > > 404 was successful. At least it doesn't report any errors beyond the > > usual LDAP connection failures at startup. > > > > The startup looks to be basically done around 2025-02-03 14:37:56 > > > > The CA returned 404's at 2025-02-03T19:39:34Z in the upgrade log. > > > > We don't have the tomcat access logs in the sosreport so we can't see > > the requests but they would likely also report 404 and no other > details. > > > > If you manually start things can you communicate with the CA? > > > > # ipactl --skip-version-check > > > > Does the CA start? If not add --ignore-service-failures > > > > Once everything else is up and settled, if the CA start failed run: > > systemctl restart pki-tomcatd@pki-tomcat > > > > And see if that is successful. I think it should succeed since it > > appears to have done so in the recent past. > > > > If so try a basic cert command: > > # openssl x509 -serial -noout -in /etc/ipa/ca.crt > > # ipa cert-show <that serial number> > > > > Does it give you data or a connection error? > > > > rob > > > > > > > > > > > On Wed, Feb 5, 2025 at 2:33 AM Alexander Bokovoy > > <aboko...@redhat.com <mailto:aboko...@redhat.com> > > > <mailto:aboko...@redhat.com <mailto:aboko...@redhat.com>>> wrote: > > > > > > On Пан, 03 лют 2025, Russell Long via FreeIPA-users wrote: > > > >Here is the log, sorry for the delay. Logs are redacted, but > the > > > only thing > > > >changed was the domain names and DNs. > > > > > > The upgrade log chokes on the CA application not being > > registered in > > > tomcat container (the corresponding /ca/rest/... path is > > giving 404 > > > error). > > > > > > So we get back to the same point as before. An upgrade has > been in > > > progress but somehow was interrupted. Directory server was > > having some > > > of listeners disabled to avoid external communication during > > the upgrade > > > and those listeners weren't recovered due to an interruption. > You > > > recovered some of them but it looks like there is still > > something that > > > messes up. > > > > > > If you are saying all services are working fine, just the > > upgrade kicks > > > in every time 'ipactl restart' is run (which is part of > > ipa.service > > > machinery), it means the logged IPA data version is older than > > what IPA > > > sees in the RPM database. Temporarily, this can be fixed by > > looking at > > > /var/lib/ipa/sysupgrade/sysupgrade.state and changing > > ipa.data_version > > > value to be exact same as the RPM package version-release > values. > > > > > > However, it would help to understand why an upgrade causes CA > > apps to > > > fail to register with the tomcat container. It looks like we > > have at > > > least three such cases on this list over past week or so, all > > on CentOS > > > 9 Stream, so there might be something? > > > > > > May be you can install sos report tool and collect a larger > > amount of > > > data altogether so that we can see a greater picture? > > > > > > # dnf install sos > > > # sos report > > --profile={identity,security,system,services,network} > > > --clean -a > > > > > > This should produce logs with consistently obfuscated > > hostnames and > > > domains across all files. You can add more domains to > > obfuscate with > > > `--domains={domain1,domain2,..}` to `sos report` tool. > > > > > > > > > > > > > > > > > > >On Wed, Jan 29, 2025 at 4:51 PM Rob Crittenden > > <rcrit...@redhat.com <mailto:rcrit...@redhat.com> > > > <mailto:rcrit...@redhat.com <mailto:rcrit...@redhat.com>>> > wrote: > > > > > > > >> Russ Long via FreeIPA-users wrote: > > > >> > Things are functional, however IPA still thinks it needs > an > > > upgrade, so > > > >> any time the service restarts, it breaks again. > > > >> > > > > >> > > > >> If you have time to run the upgrade again and send us a > > compressed > > > >> /var/log/ipaupgrade.log we can see if we can identify the > > root cause. > > > >> > > > >> rob > > > >> > > > >> > > > > > > > > > > > > > > > -- > > > / Alexander Bokovoy > > > Sr. Principal Software Engineer > > > Security / Identity Management Engineering > > > Red Hat Limited, Finland > > > > > > > -- > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue >
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
