Hi, We have a deployment of FreeIPA using 3 nodes (Master with more 2 replicas).
Recently, the master node had a problem with the process 'ns-slapd' consuming 100% of CPU. During this problem, DNS service wasn't working, IPA admin UI encountered timeout, SSH keys to access the hosts are not being loaded correctly. We observed in the logs of "dirsrv" that something related to the cachesize wasn't enough to the space needed and then ns-slapd started a process to recover it. We let the server running this operation almost one day and nothing happened. Today, we tried to: 1 - remove the failed server from the deployment, using the command below, but unfortunately, it wasn't possible to do from both the 2 other nodes. ipa-replica-manage del --force mux-idm-p03.muxi.dc --cacert=/etc/ipa/ca.crt unexpected error: cannot connect to 'ldaps://localhost.localdomain:636 2 - tried to upgrade the failed server to a most recent version of IPA using ipa-server-upgrade but it stopped in the step to connect [5/10]: starting directory server 2016-09-14T13:43:28Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2016-09-14T13:43:28Z DEBUG The ipa-server-upgrade command failed, exception: error: [Errno 111] Connection refused 2016-09-14T13:43:28Z ERROR [Errno 111] Connection refused 3 - tried to recover the 389-ds database with the command "db_recover -f -v" but nothing happened. 4 - visited similar threads but none of them helped me https://www.redhat.com/archives/freeipa-users/2013-May/msg00015.html https://www.redhat.com/archives/freeipa-users/2015-July/msg00188.html 5 - as we need to urgently recover the service, we tried to rebuild the failed server, removing and reinstalling all the packages needed by ipa-server (yum install ipa-server bind bind-dyndb-ldap ipa-server-dns) and tried to re-join the new server as a replica to receive all the data again, but it doesn't seems to work. The other nodes are working well, resolving DNS requests, allowing users to access the servers using SSH, etc. Any ideas of what I can do to rebuild the server? Versions ipa-server-4.2.0-15.0.1.el7.centos.19.x86_64 ipa-server-dns-4.2.0-15.0.1.el7.centos.19.x86_64 389-ds-base-1.3.4.0-33.el7_2.x86_64 CentOS Linux release 7.2.1511 (Core)
-- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
