Hello,
I think I have similar problem like this one, I have spent 2 days on this and I
am helpless.
ipa.service fails to start, just as ipactl start like this:
systemd[1]: Starting Identity, Policy, Audit...
ipactl[1497]: IPA version error: data needs to be upgraded (expected version
'4.8.1-4.fc31', current version '4.8.1-1.fc30')
ipactl[1497]: Automatically running upgrade, for details see
/var/log/ipaupgrade.log
ipactl[1497]: Be patient, this may take a few minutes.
ipactl[1497]: Automatic upgrade failed: Either /etc/krb5.keytab or
/etc/samba/samba.keytab are missing or unreadable
ipactl[1497]: Update complete
ipactl[1497]: Upgrading the configuration of the IPA services
ipactl[1497]: [Verifying that root certificate is published]
ipactl[1497]: [Migrate CRL publish directory]
ipactl[1497]: CRL tree already moved
ipactl[1497]: [Verifying that KDC configuration is using ipa-kdb backend]
ipactl[1497]: [Fix DS schema file syntax]
ipactl[1497]: Syntax already fixed
ipactl[1497]: [Removing RA cert from DS NSS database]
ipactl[1497]: RA cert already removed
ipactl[1497]: [Enable sidgen and extdom plugins by default]
ipactl[1497]: [Updating HTTPD service IPA configuration]
ipactl[1497]: [Updating HTTPD service IPA WSGI configuration]
ipactl[1497]: [Migrating from mod_nss to mod_ssl]
ipactl[1497]: Already migrated to mod_ssl
ipactl[1497]: [Moving HTTPD service keytab to gssproxy]
ipactl[1497]: [Removing self-signed CA]
ipactl[1497]: [Removing Dogtag 9 CA]
ipactl[1497]: [Checking for deprecated KDC configuration files]
ipactl[1497]: [Checking for deprecated backups of Samba configuration files]
ipactl[1497]: [Add missing CA DNS records]
ipactl[1497]: IPA CA DNS records already processed
ipactl[1497]: [Removing deprecated DNS configuration options]
ipactl[1497]: [Ensuring minimal number of connections]
ipactl[1497]: [Updating GSSAPI configuration in DNS]
ipactl[1497]: [Updating pid-file configuration in DNS]
ipactl[1497]: [Checking global forwarding policy in named.conf to avoid
conflicts with automatic empty zones]
ipactl[1497]: Changes to named.conf have been made, restart named
ipactl[1497]: [Upgrading CA schema]
ipactl[1497]: CA schema update complete (no changes)
ipactl[1497]: [Verifying that CA audit signing cert has 2 year validity]
ipactl[1497]: [Update certmonger certificate renewal configuration]
ipactl[1497]: Certmonger certificate renewal configuration already up-to-date
ipactl[1497]: [Enable PKIX certificate path discovery and validation]
ipactl[1497]: PKIX already enabled
ipactl[1497]: [Authorizing RA Agent to modify profiles]
ipactl[1497]: [Authorizing RA Agent to manage lightweight CAs]
ipactl[1497]: [Ensuring Lightweight CAs container exists in Dogtag database]
ipactl[1497]: [Adding default OCSP URI configuration]
ipactl[1497]: [Ensuring CA is using LDAPProfileSubsystem]
ipactl[1497]: [Migrating certificate profiles to LDAP]
ipactl[1497]: IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and
run command ipa-server-upgrade manually.
ipactl[1497]: Unexpected error - see /var/log/ipaupgrade.log for details:
ipactl[1497]: RemoteRetrieveError: Failed to authenticate to CA REST API
ipactl[1497]: The ipa-server-upgrade command failed. See
/var/log/ipaupgrade.log for more information
ipactl[1497]: See the upgrade log for more details and/or run
/usr/sbin/ipa-server-upgrade again
ipactl[1497]: Aborting ipactl
systemd[1]: ipa.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: ipa.service: Failed with result 'exit-code'.
systemd[1]: Failed to start Identity, Policy, Audit.
systemd[1]: ipa.service: Consumed 14.815s CPU time.
The main error for me is "RemoteRetrieveError: Failed to authenticate to CA
REST API". dirsrv starts successfully, then listens on socket, ports 389
udp/tcp and 636 tcp, functions as expect (I can login to server, I can login to
Web UI, ldapsearch works, etc.), even pki-tomcatd@pki-tomcat.service can be
started manually successfully, but fails with this upgrade procedure. I have
tried to troubleshoot TLS issues, but all certificates and ciphers seem OK, I
have even upgraded jss from updates-testing repository as mentioned in other
cases, but the upgrade still fails.The error from /var/log/ipaupgrade.log is:
2019-11-03T23:42:53Z DEBUG request GET
https://<ipaserver-fqdn>:8443/ca/rest/account/login
2019-11-03T23:42:53Z DEBUG request body ''
2019-11-03T23:42:54Z DEBUG response status 500
2019-11-03T23:42:54Z DEBUG response headers Content-Type:
text/html;charset=utf-8
Content-Language: en
Content-Length: 2384
Date: Sun, 03 Nov 2019 23:42:54 GMT
Connection: close
2019-11-03T23:42:54Z DEBUG response body (decoded): b'<!doctype html><html
lang="en"><head><title>HTTP Status 500 \xe2\x80\x93 Internal Server
Error</title><style type="text/css">h1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
h2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
h3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
body {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;}
b {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} p
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}
a {color:black;} a.name {color:black;} .line
{height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP
Status 500 \xe2\x80\x93 Internal Server Error</h1><hr class="line"
/><p><b>Type</b> Exception Report</p><p><b>Message</b> Subsystem
unavailable</p><p><b>Description</b> T
he server encountered an unexpected condition that prevented it from
fulfilling the
request.</p><p><b>Exception</b></p><pre>javax.ws.rs.ServiceUnavailableException:
Subsystem
unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:150)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:515)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)\n\torg.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)\n\torg.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)\n\torg.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)\n\torg.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)\n\torg.apache.tomcat.util.net.SocketProcessor
Base.run(SocketProcessorBase.java:49)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>Note</b>
The full stack trace of the root cause is available in the server logs.</p><hr
class="line" /><h3>Apache Tomcat/9.0.26</h3></body></html>'
2019-11-03T23:42:54Z ERROR IPA server upgrade failed: Inspect
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2019-11-03T23:42:54Z DEBUG File
"/usr/lib/python3.7/site-packages/ipapython/admintool.py", line 179, in execute
return_value = self.run()
File
"/usr/lib/python3.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
line 54, in run
server.upgrade()
File "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py",
line 2223, in upgrade
upgrade_configuration()
File "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py",
line 2093, in upgrade_configuration
ca_enable_ldap_profile_subsystem(ca)
File "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py",
line 414, in ca_enable_ldap_profile_subsystem
cainstance.migrate_profiles_to_ldap()
File "/usr/lib/python3.7/site-packages/ipaserver/install/cainstance.py", line
1937, in migrate_profiles_to_ldap
_create_dogtag_profile(profile_id, profile_data, overwrite=False)
File "/usr/lib/python3.7/site-packages/ipaserver/install/cainstance.py", line
1943, in _create_dogtag_profile
with api.Backend.ra_certprofile as profile_api:
File "/usr/lib/python3.7/site-packages/ipaserver/plugins/dogtag.py", line
1315, in __enter__
raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA
REST API'))
2019-11-03T23:42:54Z DEBUG The ipa-server-upgrade command failed, exception:
RemoteRetrieveError: Failed to authenticate to CA REST API
From /var/log/pki/pki-tomcat/ca/debug.2019-11-04.log the error is:
2019-11-04 20:14:47 [main] FINE: LdapBoundConnection: Connecting to
<ipaserver-fqdn>:636 with client cert auth
2019-11-04 20:14:47 [main] FINE: ldapconn/PKISocketFactory.makeSSLSocket: begins
2019-11-04 20:14:47 [main] SEVERE: Unable to create socket:
java.net.ConnectException: Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
[...]
So it seems that there is a problem with TLS and/or client certificates?
But if I restart all ipa services manually, and pki-tomcatd@pki-tomcat.service
starts succesfully, I can do any command without a problem:
$ pki -U https://srv00.ipa.stefany.eu:8443 ca-cert-find
- works OK
$ ipa cert-find
- works as well
But once I re-execute ipa-server-upgrade, pki-tomcatd@pki-tomcat.service ends
in this weird state and even pki -U ... and ipa cert-find commands stop working.
Please, can you point me into direction what should I troubleshoot further?
One weird thing that is happening, while troubleshooting according to
https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tomcatd-fails-to-start/,
as I try:
# certutil -K -d /etc/pki/pki-tomcat/alias/ -f /tmp/pwdfile.txt
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and
Certificate Services"
< 0> rsa 0ab450a1e94951591c3533a955c8421fa427ad5b caSigningCert
cert-pki-ca 0e8ab82d-a531-4bf5-8692-6981852ce926
< 1> rsa 3b81657ea1a630eaf10cf4e1fd0a9dd8a6ba611e NSS Certificate
DB:caSigningCert cert-pki-ca
< 2> rsa fd88a1210bb455802c439eef684c772925aea168 NSS Certificate
DB:ocspSigningCert cert-pki-ca
< 3> rsa 5258b246ec86056690dd1123594a2db68e30d8fc NSS Certificate
DB:subsystemCert cert-pki-ca
< 4> rsa 679e58688f3cb09b979e67b03dbc1e242b99d11c NSS Certificate
DB:auditSigningCert cert-pki-ca
< 5> rsa d2a930334d00dd67ae5530b34169a5683b987c5d NSS Certificate
DB:Server-Cert cert-pki-ca
< 6> rsa 3fe989c7f02327de578c8a39e6c4f9f2d02dd56a (orphan)
then it works OK, but if I try using alias, then I get this error:
# certutil -K -d /etc/pki/pki-tomcat/alias/ -f /tmp/pwdfile.txt -n "NSS
Certificate DB:subsystemCert cert-pki-ca"
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and
Certificate Services"
certutil: problem listing keys: SEC_ERROR_INVALID_ARGS: security library:
invalid arguments.
And I have tried using different quotes, with "NSS Certificate DB:" or without,
it always fails.
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
