To answer your question, yes, /etc/named/ipa-ext.conf and /etc/named/ipa-options-ext.conf exist.
When I attempted to start named-pkcs11.service. It failed. Journalctl initially said there were issues with selinux. Anyhow, I attempted to start the service again after making the selinux policy entries that were suggested. I still was unable to get the service to start. Though, this time I didn't get any selinux messages. Here is what happened at the first start of named-pkcs11.service just for reference: [root@utility ~]# journalctl -xe You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'ipa-dnskeysync-' --raw | audit2allow -M my-ipadnskeysync # semodule -X 300 -i my-ipadnskeysync.pp Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: AnalyzeThread.run(): Set alarm timeout to 10 Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: AnalyzeThread.run(): Cancel pending alarm Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: SELinux is preventing /usr/libexec/platform-python3.6 from 'read, write' acce> Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: SELinux is preventing /usr/libexec/platform-python3.6 from 'read, write' acce> ***** Plugin catchall (100. confidence) suggests ************************** If you believe that platform-python3.6 should be allowed read write access on> Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'ipa-dnskeysync-' --raw | audit2allow -M my-ipadnskeysync # semodule -X 300 -i my-ipadnskeysync.pp Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: AnalyzeThread.run(): Set alarm timeout to 10 Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: AnalyzeThread.run(): Cancel pending alarm Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: SELinux is preventing /usr/libexec/platform-python3.6 from lock access on the> Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: SELinux is preventing /usr/libexec/platform-python3.6 from lock access on the> ***** Plugin catchall (100. confidence) suggests ************************** If you believe that platform-python3.6 should be allowed lock access on the g> Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'ipa-dnskeysync-' --raw | audit2allow -M my-ipadnskeysync # semodule -X 300 -i my-ipadnskeysync.pp Aug 30 07:10:49 utility.idm.nac-issa.org setroubleshoot[21841]: AnalyzeThread.run(): Set alarm timeout to 10 Here is the 2nd run after making the selinux entries. [root@utility ~]# systemctl start named-pkcs11.service Job for named-pkcs11.service failed because the control process exited with error code. See "systemctl status named-pkcs11.service" and "journalctl -xe" for details. [root@utility ~]# journalctl -xe Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: built with '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '> Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: running as: named-pkcs11 -u named -c /etc/named.conf Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: compiled by GCC 8.4.1 20200928 (Red Hat 8.4.1-1) Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: compiled with libxml2 version: 2.9.7 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: linked to libxml2 version: 20907 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: compiled with libjson-c version: 0.13.1 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: linked to libjson-c version: 0.13.1 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: compiled with zlib version: 1.2.11 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: linked to zlib version: 1.2.11 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: threads support is enabled Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: ---------------------------------------------------- Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: BIND 9 is maintained by Internet Systems Consortium, Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: Inc. (ISC), a non-profit 501(c)(3) public-benefit Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: corporation. Support and training for BIND 9 are Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: available at https://www.isc.org/support Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: ---------------------------------------------------- Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: adjusted limit on open files from 262144 to 1048576 Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: found 4 CPUs, using 4 worker threads Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: using 3 UDP listeners per interface Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: using up to 21000 sockets Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: initializing DST: PKCS#11 initialization failed Aug 30 07:15:51 utility.idm.nac-issa.org named-pkcs11[22071]: exiting (due to fatal error) Aug 30 07:15:51 utility.idm.nac-issa.org systemd[1]: named-pkcs11.service: Control process exited, code=exited status=1 Aug 30 07:15:51 utility.idm.nac-issa.org systemd[1]: named-pkcs11.service: Failed with result 'exit-code'. -- Subject: Unit failed -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- The unit named-pkcs11.service has entered the 'failed' state with result 'exit-code'. Aug 30 07:15:51 utility.idm.nac-issa.org systemd[1]: Failed to start Berkeley Internet Name Domain (DNS) with native PKCS#11. -- Subject: Unit named-pkcs11.service has failed -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- Unit named-pkcs11.service has failed. -- -- The result is failed. [root@utility ~]# cat /etc/named/ipa-ext.conf // Custom managed file. // Here you can set your own options, for instance ACL for recursion access: // // acl "trusted_network" { // localnets; // localhost; // 234.234.234.0/24; // 2001::co:ffee:babe:1/48; // }; // options { // allow-recursion {trusted_network;}; // allow-query-cache {trusted_network;}; // }; // // This file will NOT be overridden during updates! [root@utility ~]# cat /etc/named/ipa-options-ext.conf /* User customization for BIND named * * This file is included in /etc/named.conf and is not modified during IPA * upgrades. * * It must only contain "options" settings. Any other setting must be * configured in /etc/named/ipa-ext.conf. * * Examples: * allow-recursion { trusted_network; }; * allow-query-cache { trusted_network; }; */ /* turns on IPv6 for port 53, IPv4 is on by default for all ifaces */ listen-on-v6 { any; }; /* dnssec-enable is obsolete and 'yes' by default */ dnssec-validation yes; [root@utility data]# systemctl status named-pkcs11.service ● named-pkcs11.service - Berkeley Internet Name Domain (DNS) with native PKCS#11 Loaded: loaded (/usr/lib/systemd/system/named-pkcs11.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Mon 2021-08-30 07:27:50 CDT; 4min 49s ago Process: 22249 ExecStart=/usr/sbin/named-pkcs11 -u named -c ${NAMEDCONF} $OPTIONS (code=exited, status=1/FAILURE) Process: 22244 ExecStartPre=/bin/bash -c if [ ! "$DISABLE_ZONE_CHECKING" == "yes" ]; then /usr/sbin/named-checkconf -z "$NAMEDCONF"; else e> Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: ---------------------------------------------------- Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: adjusted limit on open files from 262144 to 1048576 Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: found 4 CPUs, using 4 worker threads Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: using 3 UDP listeners per interface Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: using up to 21000 sockets Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: initializing DST: PKCS#11 initialization failed Aug 30 07:27:50 utility.idm.nac-issa.org named-pkcs11[22250]: exiting (due to fatal error) Aug 30 07:27:50 utility.idm.nac-issa.org systemd[1]: named-pkcs11.service: Control process exited, code=exited status=1 Aug 30 07:27:50 utility.idm.nac-issa.org systemd[1]: named-pkcs11.service: Failed with result 'exit-code'. Aug 30 07:27:50 utility.idm.nac-issa.org systemd[1]: Failed to start Berkeley Internet Name Domain (DNS) with native PKCS#11. [root@utility data]# journalctl -xe Aug 30 07:27:53 utility.idm.nac-issa.org systemd[1]: Stopped PKI Tomcat Server pki-tomcat. -- Subject: Unit pki-tomcatd@pki-tomcat.service has finished shutting down -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- Unit pki-tomcatd@pki-tomcat.service has finished shutting down. Aug 30 07:27:54 utility.idm.nac-issa.org ns-slapd[1665]: [30/Aug/2021:07:27:54.054683013 -0500] - INFO - bdb_pre_close - Waiting for 4 databa> Aug 30 07:27:55 utility.idm.nac-issa.org ns-slapd[1665]: [30/Aug/2021:07:27:55.032053458 -0500] - INFO - bdb_pre_close - All database threads> Aug 30 07:27:55 utility.idm.nac-issa.org named[1527]: LDAP error: Can't contact LDAP server: ldap_sync_poll() failed Aug 30 07:27:55 utility.idm.nac-issa.org named[1527]: ldap_syncrepl will reconnect in 60 seconds Aug 30 07:27:55 utility.idm.nac-issa.org ns-slapd[1665]: [30/Aug/2021:07:27:55.054454093 -0500] - INFO - ldbm_back_instance_set_destructor - > Aug 30 07:27:55 utility.idm.nac-issa.org ns-slapd[1665]: [30/Aug/2021:07:27:55.057417960 -0500] - INFO - connection_post_shutdown_cleanup - s> Aug 30 07:27:55 utility.idm.nac-issa.org ns-slapd[1665]: [30/Aug/2021:07:27:55.059926010 -0500] - INFO - main - slapd stopped. Aug 30 07:27:55 utility.idm.nac-issa.org systemd[1]: dirsrv@IDM-NAC-ISSA-ORG.service: Succeeded. -- Subject: Unit succeeded -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- The unit dirsrv@IDM-NAC-ISSA-ORG.service has successfully entered the 'dead' state. Aug 30 07:27:55 utility.idm.nac-issa.org systemd[1]: Stopped 389 Directory Server IDM-NAC-ISSA-ORG.. -- Subject: Unit dirsrv@IDM-NAC-ISSA-ORG.service has finished shutting down -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- Unit dirsrv@IDM-NAC-ISSA-ORG.service has finished shutting down. Aug 30 07:27:59 utility.idm.nac-issa.org named[1527]: network unreachable resolving 'a-ups-presencecore4-prod-azsc.eastus2.cloudapp.azure.com> Aug 30 07:27:59 utility.idm.nac-issa.org named[1527]: network unreachable resolving 'a-ups-presencecore4-prod-azsc.eastus2.cloudapp.azure.com> Aug 30 07:28:55 utility.idm.nac-issa.org named[1527]: LDAP error: Can't contact LDAP server: bind to LDAP server failed Aug 30 07:28:55 utility.idm.nac-issa.org named[1527]: ldap_syncrepl will reconnect in 60 seconds Aug 30 07:29:55 utility.idm.nac-issa.org named[1527]: LDAP error: Can't contact LDAP server: bind to LDAP server failed Aug 30 07:29:55 utility.idm.nac-issa.org named[1527]: ldap_syncrepl will reconnect in 60 seconds Aug 30 07:30:55 utility.idm.nac-issa.org named[1527]: LDAP error: Can't contact LDAP server: bind to LDAP server failed Aug 30 07:30:55 utility.idm.nac-issa.org named[1527]: ldap_syncrepl will reconnect in 60 seconds Aug 30 07:31:55 utility.idm.nac-issa.org named[1527]: LDAP error: Can't contact LDAP server: bind to LDAP server failed Aug 30 07:31:55 utility.idm.nac-issa.org named[1527]: ldap_syncrepl will reconnect in 60 seconds Aug 30 07:32:55 utility.idm.nac-issa.org named[1527]: LDAP error: Can't contact LDAP server: bind to LDAP server failed Aug 30 07:32:55 utility.idm.nac-issa.org named[1527]: ldap_syncrepl will reconnect in 60 seconds It looks like I need to troubleshoot section 4 further.. auth_method, sasl_mech, sasl_user, all seem to be present in my /etc/named.conf file I was unable to find bind_dn, password, sasl_realm, sasl_password and krb5_principal. [root@utility data]# cat /etc/named.conf options { // turns on IPv6 for port 53, IPv4 is on by default for all ifaces #listen-on-v6 {any;}; // Put files that named is allowed to write in the data/ directory: directory "/var/named"; // the default dump-file "data/cache_dump.db"; statistics-file "data/named_stats.txt"; memstatistics-file "data/named_mem_stats.txt"; // If not explicitly set, the ACLs for "allow-query-cache" and // "allow-recursion" are set to "localnets; localhost;". // If either "allow-query-cache" or "allow-recursion" is set, // the other would be set the same value. // Please refer to /etc/named/ipa-ext.conf // for more information tkey-gssapi-keytab "/etc/named.keytab"; pid-file "/run/named/named.pid"; dnssec-enable yes; dnssec-validation yes; /* Path to ISC DLV key */ bindkeys-file "/etc/named.iscdlv.key"; managed-keys-directory "/var/named/dynamic"; /* crypto policy snippet on platforms with system-wide policy. */ // not available }; /* If you want to enable debugging, eg. using the 'rndc trace' command, * By default, SELinux policy does not allow named to modify the /var/named directory, * so put the default debug log file in data/ : */ logging { channel default_debug { file "data/named.run"; severity dynamic; print-time yes; }; }; zone "." IN { type hint; file "named.ca"; }; include "/etc/named.rfc1912.zones"; include "/etc/named.root.key"; /* custom configuration snippet */ include "/etc/named/ipa-ext.conf"; /* WARNING: This part of the config file is IPA-managed. * Modifications may break IPA setup or upgrades. */ dyndb "ipa" "/usr/lib64/bind/ldap.so" { uri "ldapi://%2fvar%2frun%2fslapd-IDM-NAC-ISSA-ORG.socket"; base "cn=dns,dc=idm,dc=nac-issa,dc=org"; server_id "utility.idm.nac-issa.org"; auth_method "sasl"; sasl_mech "GSSAPI"; sasl_user "DNS/utility.idm.nac-issa.org"; }; /* End of IPA-managed part. */ ________________________________ From: Florence Renaud <f...@redhat.com> Sent: Monday, August 30, 2021 2:39 AM To: FreeIPA users list <freeipa-users@lists.fedorahosted.org> Cc: Rob Crittenden <rcrit...@redhat.com>; Jeremy Tourville <jeremy_tourvi...@hotmail.com> Subject: Re: [Freeipa-users] Re: Unable to start directory server after updates Hi, on rhel8, IPA is using named-pkcs11.service, not named.service. In order to manually start the bind service, you would need to use "systemctl start named-pkcs11.service". The journal may contain additional logs, as well as the output of "systemctl status named-pkcs11.service". IIRC in ipa 4.9, ipa introduced bind configuration snippets in /etc/named/ipa-ext.conf and /etc/named/ipa-options-ext.conf. Do you have such configuration files? flo On Sun, Aug 29, 2021 at 3:45 PM Jeremy Tourville via FreeIPA-users <freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org>> wrote: I found this page on troubleshooting - https://docs.pagure.org/bind-dyndb-ldap/BIND9/NamedCannotStart.html I can manually start named.service but cannot start named when using ipactl. Section 1 I was able to get a log (this log is prior to changes made in section 4) #less /var/named/data/named.run reloading configuration succeeded reloading zones succeeded network unreachable resolving './DNSKEY/IN': 2001:7fd::1#53 network unreachable resolving './DNSKEY/IN': 2001:500:1::53#53 network unreachable resolving './DNSKEY/IN': 2001:500:9f::42#53 network unreachable resolving './DNSKEY/IN': 2001:500:2f::f#53 network unreachable resolving './DNSKEY/IN': 2001:500:12::d0d#53 network unreachable resolving './DNSKEY/IN': 2001:500:200::b#53 network unreachable resolving './DNSKEY/IN': 2001:503:ba3e::2:30#53 network unreachable resolving './DNSKEY/IN': 2001:500:a8::e#53 network unreachable resolving './DNSKEY/IN': 2001:7fe::53#53 network unreachable resolving './DNSKEY/IN': 2001:dc3::35#53 network unreachable resolving './DNSKEY/IN': 2001:500:2::c#53 network unreachable resolving './DNSKEY/IN': 2001:500:2d::d#53 network unreachable resolving './DNSKEY/IN': 2001:503:c27::2:30#53 all zones loaded running managed-keys-zone: Key 20326 for zone . acceptance timer complete: key now trusted With the changes in section 4 (below) I now see this additional info in the log: received control channel command 'stop' shutting down: flushing changes stopping command channel on 127.0.0.1#953 stopping command channel on ::1#953 no longer listening on 127.0.0.1#53 no longer listening on ::1#53 exiting I was unable to get a log from tmp/named_krb5.log using the rhel/fedora method. Do I need to use the archlinux method? Section 2 I don't see any evidence of this issue based on logs. Furthermore, hostname FQDN and /etc/hosts are set properly according to the examples shown Section 3 The values here match Section 4 I see that my system was running a named.conf file that didn't have any credentials. I looked at my yum history and the timestamps for my named.conf* files. The yum update that most likely affected them was run at 9:52. The two oldest files are marked 9:55 and I presume are the backups as part of the update process. [root@utility etc]# ls -la named.conf* -rw-r-----. 1 root named 1876 Aug 29 08:01 named.conf -rw-r-----. 1 root named 1705 May 27 15:49 named.conf.bak -rw-r--r--. 1 root root 1876 Aug 28 09:55 named.conf.ipa-backup -rw-r-----. 1 root named 1535 Aug 28 09:55 named.conf.rpmsave I did attempt to copy the oldest files over the existing named.conf and start the named service. I still didn't have any luck in either case. #cp named.conf.rpmsave named.conf #ipactl start #cp named.conf.ipa-backup named.conf #ipactl start Systemctl status when using named.conf.rpmsave version: [root@utility etc]# systemctl status named ● named.service - Berkeley Internet Name Domain (DNS) Loaded: loaded (/usr/lib/systemd/system/named.service; linked; vendor preset: disabled) Active: active (running) since Sun 2021-08-29 08:38:05 CDT; 1s ago Process: 2294 ExecStart=/usr/sbin/named -u named -c ${NAMEDCONF} $OPTIONS (code=exited, status=0/SUCCESS) Process: 2291 ExecStartPre=/bin/bash -c if [ ! "$DISABLE_ZONE_CHECKING" == "yes" ]; then /usr/sbin/named-checkconf -z "$NAMEDCONF"; else ec> Main PID: 2296 (named) Tasks: 8 (limit: 37317) Memory: 59.5M CGroup: /system.slice/named.service └─2296 /usr/sbin/named -u named -c /etc/named.conf Aug 29 08:38:05 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: managed-keys-zone: Key 20326 for zone . acceptance timer complete: key now trusted Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: resolver priming query complete Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: LDAP configuration synchronization failed: socket is not connected Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: ldap_syncrepl will reconnect in 60 seconds Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: network unreachable resolving '_ldap._tcp.idm.nac-issa.org/SRV/IN<http://tcp.idm.nac-issa.org/SRV/IN>': 2001:500:f::1#53 Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: network unreachable resolving '_ldap._tcp.idm.nac-issa.org/SRV/IN<http://tcp.idm.nac-issa.org/SRV/IN>': 2001:500:c::1#53 Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: network unreachable resolving '_ldap._tcp.idm.nac-issa.org/SRV/IN<http://tcp.idm.nac-issa.org/SRV/IN>': 2001:500:40::1#53 Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: network unreachable resolving '_ldap._tcp.idm.nac-issa.org/SRV/IN<http://tcp.idm.nac-issa.org/SRV/IN>': 2001:500:48::1#53 Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: network unreachable resolving '_ldap._tcp.idm.nac-issa.org/SRV/IN<http://tcp.idm.nac-issa.org/SRV/IN>': 2001:500:b::1#53 Aug 29 08:38:06 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2296]: network unreachable resolving '_ldap._tcp.idm.nac-issa.org/SRV/IN<http://tcp.idm.nac-issa.org/SRV/IN>': 2001:500:e::1#53 Systemctl status when using named.conf.ipa-backup version: [root@utility etc]# systemctl start named [root@utility etc]# systemctl status named ● named.service - Berkeley Internet Name Domain (DNS) Loaded: loaded (/usr/lib/systemd/system/named.service; linked; vendor preset: disabled) Active: active (running) since Sun 2021-08-29 08:33:54 CDT; 5s ago Process: 2251 ExecStart=/usr/sbin/named -u named -c ${NAMEDCONF} $OPTIONS (code=exited, status=0/SUCCESS) Process: 2247 ExecStartPre=/bin/bash -c if [ ! "$DISABLE_ZONE_CHECKING" == "yes" ]; then /usr/sbin/named-checkconf -z "$NAMEDCONF"; else ec> Main PID: 2252 (named) Tasks: 8 (limit: 37317) Memory: 64.7M CGroup: /system.slice/named.service └─2252 /usr/sbin/named -u named -c /etc/named.conf Aug 29 08:33:55 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'eur2.akam.net/AAAA/IN<http://eur2.akam.net/AAAA/IN>': 2600:1401:1::43#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube2.idm.nac-issa.org/AAAA/IN<http://kube2.idm.nac-issa.org/AAAA/IN>': 2a00:edc0:107::1#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube2.idm.nac-issa.org/AAAA/IN<http://kube2.idm.nac-issa.org/AAAA/IN>': 2a00:edc0:107::49#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube2.idm.nac-issa.org/AAAA/IN<http://kube2.idm.nac-issa.org/AAAA/IN>': 2402:cf80:107::1#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube2.idm.nac-issa.org/AAAA/IN<http://kube2.idm.nac-issa.org/AAAA/IN>': 2402:cf80:107::49#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'nac-issa.org/DS/IN<http://nac-issa.org/DS/IN>': 2001:500:c::1#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube1.idm.nac-issa.org/A/IN<http://kube1.idm.nac-issa.org/A/IN>': 2402:cf80:107::1#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube1.idm.nac-issa.org/AAAA/IN<http://kube1.idm.nac-issa.org/AAAA/IN>': 2402:cf80:107::1#53 Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube3.idm.nac-issa.org.idm.nac-issa.org/A/IN<http://kube3.idm.nac-issa.org.idm.nac-issa.org/A/IN>': 2402:cf80> Aug 29 08:33:56 utility.idm.nac-issa.org<http://utility.idm.nac-issa.org> named[2252]: network unreachable resolving 'kube3.idm.nac-issa.org.idm.nac-issa.org/AAAA/IN<http://kube3.idm.nac-issa.org.idm.nac-issa.org/AAAA/IN>': 2402:c> Here are the contents of my file: #less /etc/named.conf (named.conf.rpm version) options { // turns on IPv6 for port 53, IPv4 is on by default for all ifaces listen-on-v6 {any;}; // Put files that named is allowed to write in the data/ directory: directory "/var/named"; // the default dump-file "data/cache_dump.db"; statistics-file "data/named_stats.txt"; memstatistics-file "data/named_mem_stats.txt"; // If not explicitly set, the ACLs for "allow-query-cache" and // "allow-recursion" are set to "localnets; localhost;". // If either "allow-query-cache" or "allow-recursion" is set, // the other would be set the same value. // Please refer to /etc/named/ipa-ext.conf // for more informations tkey-gssapi-keytab "/etc/named.keytab"; pid-file "/run/named/named.pid"; dnssec-enable yes; dnssec-validation yes; /* Path to ISC DLV key */ bindkeys-file "/etc/named.iscdlv.key"; managed-keys-directory "/var/named/dynamic"; /* crypto policy snippet on platforms with system-wide policy. */ // not available }; /* If you want to enable debugging, eg. using the 'rndc trace' command, * By default, SELinux policy does not allow named to modify the /var/named directory, * so put the default debug log file in data/ : */ logging { channel default_debug { file "data/named.run"; severity dynamic; print-time yes; }; }; zone "." IN { type hint; file "named.ca<http://named.ca>"; }; include "/etc/named.rfc1912.zones"; include "/etc/named.root.key"; /* custom configuration snippet */ include "/etc/named/ipa-ext.conf"; /* WARNING: This part of the config file is IPA-managed. * Modifications may break IPA setup or upgrades. */ dyndb "ipa" "/usr/lib64/bind/ldap.so" { uri "ldapi://%2fvar%2frun%2fslapd-IDM-NAC-ISSA-ORG.socket"; base "cn=dns, dc=idm,dc=nac-issa,dc=org"; server_id "utility.idm.nac-issa.org<http://utility.idm.nac-issa.org>"; auth_method "sasl"; sasl_mech "GSSAPI"; sasl_user "DNS/utility.idm.nac-issa.org<http://utility.idm.nac-issa.org>"; }; /* End of IPA-managed part. */ I also compared the two oldest files but I am not sure what changes should be made in my existing named.conf. # diff named.conf.rpmsave named.conf.ipa-backup 1,9d0 < /* WARNING: This config file is managed by IPA. < * < * DO NOT MODIFY! Any modification will be overwritten by upgrades. < * < * < * - /etc/named/ipa-options-ext.conf (for options) < * - /etc/named/ipa-ext.conf (all other settings) < */ < 10a2,4 > // turns on IPv6 for port 53, IPv4 is on by default for all ifaces > listen-on-v6 {any;}; > 17c11,16 < tkey-gssapi-keytab "/etc/named.keytab"; --- > // If not explicitly set, the ACLs for "allow-query-cache" and > // "allow-recursion" are set to "localnets; localhost;". > // If either "allow-query-cache" or "allow-recursion" is set, > // the other would be set the same value. > // Please refer to /etc/named/ipa-ext.conf > // for more informations 18a18 > tkey-gssapi-keytab "/etc/named.keytab"; 21c21,25 < managed-keys-directory "/var/named/dynamic"; --- > dnssec-enable yes; > dnssec-validation yes; > > /* Path to ISC DLV key */ > bindkeys-file "/etc/named.iscdlv.key"; 23,24c27 < /* user customizations of options */ < include "/etc/named/ipa-options-ext.conf"; --- > managed-keys-directory "/var/named/dynamic"; 50c53 < /* user customization */ --- > /* custom configuration snippet */ 52a56,58 > /* WARNING: This part of the config file is IPA-managed. > * Modifications may break IPA setup or upgrades. > */ 55c61 < base "cn=dns,dc=idm,dc=nac-issa,dc=org"; --- > base "cn=dns, dc=idm,dc=nac-issa,dc=org"; 60a67 > /* End of IPA-managed part. */ ________________________________ From: Jeremy Tourville <jeremy_tourvi...@hotmail.com<mailto:jeremy_tourvi...@hotmail.com>> Sent: Saturday, August 28, 2021 7:07 PM To: freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org> <freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org>> Cc: Rob Crittenden <rcrit...@redhat.com<mailto:rcrit...@redhat.com>> Subject: Re: [Freeipa-users] Unable to start directory server after updates OK, I quickly realized I couldn't yum/dnf downgrade as I still had a version/data mismatch. Now I understand what the error means. I did the latter part of my previous question and performed an ipa-server-upgrade. .... ..... The IPA services were upgraded The ipa-server-upgrade command was successful Now I tried to start my ipa server but had limited success. Named service won't start .... .... Starting named Service Failed to start named Service Shutting down I tried to force and see what else would have issues #ipactl start --ignore-service-failure .... .... Failed to start named Service Forced start, ignoring named Service, continuing normal operation .... .... Starting ipa-dnskeysyncd Service Failed to start ipa-dnskeysyncd Service Forced start, ignoring ipa-dnskeysyncd Service, continuing normal operation ipa: INFO: The ipactl command was successful Here is the entire sequence- [root@utility slapd-IDM-NAC-ISSA-ORG]# ipa-server-upgrade Upgrading IPA:. Estimated time: 1 minute 30 seconds [1/9]: saving configuration [2/9]: disabling listeners [3/9]: enabling DS global lock [4/9]: disabling Schema Compat [5/9]: starting directory server [6/9]: updating schema [7/9]: upgrading server [8/9]: stopping directory server [9/9]: restoring configuration Done. Update complete Upgrading IPA services Upgrading the configuration of the IPA services Disabled p11-kit-proxy [Verifying that root certificate is published] [Migrate CRL publish directory] CRL tree already moved [Verifying that KDC configuration is using ipa-kdb backend] [Fix DS schema file syntax] Syntax already fixed [Removing RA cert from DS NSS database] RA cert already removed [Enable sidgen and extdom plugins by default] [Updating HTTPD service IPA configuration] [Updating HTTPD service IPA WSGI configuration] Nothing to do for configure_httpd_wsgi_conf [Migrating from mod_nss to mod_ssl] Already migrated to mod_ssl [Moving HTTPD service keytab to gssproxy] [Removing self-signed CA] [Removing Dogtag 9 CA] [Checking for deprecated KDC configuration files] [Checking for deprecated backups of Samba configuration files] [Remove FILE: prefix from 'dedicated keytab file' in Samba configuration] [Update 'max smbd processes' in Samba configuration to prevent unlimited SMBLoris attack amplification] dnssec-validation yes [Add missing CA DNS records] IPA CA DNS records already processed DNS service is not configured [Upgrading CA schema] CA schema update complete [Update certmonger certificate renewal configuration] Certmonger certificate renewal configuration already up-to-date [Enable PKIX certificate path discovery and validation] PKIX already enabled [Authorizing RA Agent to modify profiles] [Authorizing RA Agent to manage lightweight CAs] [Ensuring Lightweight CAs container exists in Dogtag database] [Adding default OCSP URI configuration] [Disabling cert publishing] pki-tomcat configuration changed, restart pki-tomcat [Ensuring CA is using LDAPProfileSubsystem] [Migrating certificate profiles to LDAP] Migrating profile 'caECServerCertWithSCT' Migrating profile 'caServerCertWithSCT' Migrating profile 'caServerKeygen_DirUserCert' Migrating profile 'caServerKeygen_UserCert' [Ensuring presence of included profiles] [Add default CA ACL] Default CA ACL already added [Updating ACME configuration] [Migrating to authselect profile] Already migrated to authselect profile [Create systemd-user hbac service and rule] hbac service systemd-user already exists [Add r...@idm.nac-issa.org<mailto:r...@idm.nac-issa.org> alias to admin account] Alias already exists [Setup SPAKE] [Setup PKINIT] [Enable server krb5.conf snippet] [Adding ipa-ca alias to HTTP certificate] Resubmitting HTTP cert tracking request The IPA services were upgraded The ipa-server-upgrade command was successful [root@utility slapd-IDM-NAC-ISSA-ORG]# ipactl start Existing service file detected! Assuming stale, cleaning and proceeding Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Failed to start named Service Shutting down Hint: You can use --ignore-service-failure option for forced start in case that a non-critical service failed Aborting ipactl [root@utility slapd-IDM-NAC-ISSA-ORG]# ipactl start --ignore-service-failure Existing service file detected! Assuming stale, cleaning and proceeding Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Failed to start named Service Forced start, ignoring named Service, continuing normal operation Starting httpd Service Starting ipa-custodia Service Starting pki-tomcatd Service Starting smb Service Starting winbind Service Starting ipa-otpd Service Starting ipa-dnskeysyncd Service Failed to start ipa-dnskeysyncd Service Forced start, ignoring ipa-dnskeysyncd Service, continuing normal operation ipa: INFO: The ipactl command was successful [root@utility slapd-IDM-NAC-ISSA-ORG]# ________________________________ From: Jeremy Tourville <jeremy_tourvi...@hotmail.com<mailto:jeremy_tourvi...@hotmail.com>> Sent: Saturday, August 28, 2021 6:45 PM To: freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org> <freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org>> Cc: Rob Crittenden <rcrit...@redhat.com<mailto:rcrit...@redhat.com>> Subject: Re: [Freeipa-users] Unable to start directory server after updates CentOS Linux release 8.4.2105 VERSION: 4.9.2, API_VERSION: 2.240 Prior to any updates I was at ver 8.2 of CentOS The shared library was loaded and now I can start dirsrv. THANKS! That's definitely big a step in the right direction. As I thought, my upgrade looks like it caused the version be too new for the existing dirsrv data. I thought I had set my OS distro release version and that is my own dumb mistake... IPA version error: data needs to be upgraded (expected version '4.9.2-4.module_el8.4.0+846+96522ed7', current version '4.8.4-7.module_el8.2.0+374+0d2d74a1') I am thinking I could downgrade to get things up and running or do you suggest upgrading the data to work with the application version I have installed? [root@utility slapd-IDM-NAC-ISSA-ORG]# ipactl status Directory Service: RUNNING krb5kdc Service: STOPPED kadmin Service: STOPPED named Service: STOPPED httpd Service: STOPPED ipa-custodia Service: STOPPED pki-tomcatd Service: STOPPED smb Service: STOPPED winbind Service: STOPPED ipa-otpd Service: STOPPED ipa-dnskeysyncd Service: STOPPED 9 service(s) are not running [root@utility slapd-IDM-NAC-ISSA-ORG]# ipactl start IPA version error: data needs to be upgraded (expected version '4.9.2-4.module_el8.4.0+846+96522ed7', current version '4.8.4-7.module_el8.2.0+374+0d2d74a1') Automatically running upgrade, for details see /var/log/ipaupgrade.log Be patient, this may take a few minutes. Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Failed to start named Service Shutting down Hint: You can use --ignore-service-failure option for forced start in case that a non-critical service failed Aborting ipactl ________________________________ From: Rob Crittenden <rcrit...@redhat.com<mailto:rcrit...@redhat.com>> Sent: Saturday, August 28, 2021 5:31 PM To: FreeIPA users list <freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org>> Cc: Jeremy Tourville <jeremy_tourvi...@hotmail.com<mailto:jeremy_tourvi...@hotmail.com>> Subject: Re: [Freeipa-users] Unable to start directory server after updates Jeremy Tourville via FreeIPA-users wrote: > I was doing some maintenance and updates this morning. At some point I > noticed I couldn't reach the web interface anymore. My server has been up > and running for the last year and is not a new install. I reviewed > //var/log/dirsrv/slapd-IDM-NAC-ISSA-ORG/errors. I also confirmed I did not > have disk space issues. > > Here is part of my log file: > [28/Aug/2021:10:46:35.380380540 -0500] - INFO - slapd_daemon - slapd started. > Listening on All Interfaces port 389 for LDAP requests > [28/Aug/2021:10:46:35.383040751 -0500] - INFO - slapd_daemon - Listening on > All Interfaces port 636 for LDAPS requests > [28/Aug/2021:10:46:35.385415998 -0500] - INFO - slapd_daemon - Listening on > /var/run/slapd-IDM-NAC-ISSA-ORG.socket for LDAPI requests > [28/Aug/2021:10:46:35.439358079 -0500] - ERR - schema-compat-plugin - > schema-compat-plugin tree scan will start in about 5 seconds! > [28/Aug/2021:10:46:40.494600578 -0500] - WARN - str2entry_dupcheck - > Duplicate value for attribute type memberUid detected in entry > cn=sudo-infra,cn=groups,cn=compat,dc=idm,dc=nac-issa,dc=org. Extra value > ignored. > [28/Aug/2021:10:46:40.527665958 -0500] - WARN - str2entry_dupcheck - > Duplicate value for attribute type memberUid detected in entry > cn=sudo-devel,cn=groups,cn=compat,dc=idm,dc=nac-issa,dc=org. Extra value > ignored. > [28/Aug/2021:10:46:40.560185359 -0500] - ERR - schema-compat-plugin - > warning: no entries set up under cn=computers, > cn=compat,dc=idm,dc=nac-issa,dc=org > [28/Aug/2021:10:46:40.582782578 -0500] - ERR - schema-compat-plugin - > Finished plugin initialization. > [28/Aug/2021:11:20:49.697931599 -0500] - INFO - op_thread_cleanup - slapd > shutting down - signaling operation threads - op stack size 4 max work q size > 2 max work q stack size 2 > [28/Aug/2021:11:20:49.706989092 -0500] - INFO - slapd_daemon - slapd shutting > down - closing down internal subsystems and plugins > [28/Aug/2021:11:20:49.724450159 -0500] - INFO - bdb_pre_close - Waiting for 4 > database threads to stop > [28/Aug/2021:11:20:51.131059518 -0500] - INFO - bdb_pre_close - All database > threads now stopped > [28/Aug/2021:11:20:51.152587508 -0500] - INFO - > ldbm_back_instance_set_destructor - Set of instances destroyed > [28/Aug/2021:11:20:51.155514615 -0500] - INFO - > connection_post_shutdown_cleanup - slapd shutting down - freed 2 work q stack > objects - freed 7 op stack objects > [28/Aug/2021:11:20:51.158002944 -0500] - INFO - main - slapd stopped. > [28/Aug/2021:13:14:20.585994349 -0500] - NOTICE - config_set_port - > Non-Secure Port Disabled > [28/Aug/2021:13:14:20.607117053 -0500] - ERR - symload_report_error - > Netscape Portable Runtime error -5977: > /usr/lib64/dirsrv/plugins/libipa_cldap.so: cannot open shared object file: No > such file or directory > [28/Aug/2021:13:14:20.609768545 -0500] - ERR - symload_report_error - Could > not open library "/usr/lib64/dirsrv/plugins/libipa_cldap.so" for plugin > ipa_cldap > [28/Aug/2021:13:14:20.612257544 -0500] - ERR - load_plugin_entry - Unable to > load plugin "cn=ipa_cldap,cn=plugins,cn=config" > [28/Aug/2021:13:14:21.012890173 -0500] - ERR - symload_report_error - > Netscape Portable Runtime error -5977: > /usr/lib64/dirsrv/plugins/libipa_cldap.so: cannot open shared object file: No > such file or directory > [28/Aug/2021:13:14:21.018097465 -0500] - ERR - symload_report_error - Could > not open library "/usr/lib64/dirsrv/plugins/libipa_cldap.so" for plugin > ipa_cldap > [28/Aug/2021:13:14:21.020655816 -0500] - ERR - load_plugin_entry - Unable to > load plugin "cn=ipa_cldap,cn=plugins,cn=config" > [28/Aug/2021:13:15:53.219524942 -0500] - ERR - symload_report_error - > Netscape Portable Runtime error -5977: > /usr/lib64/dirsrv/plugins/libipa_cldap.so: cannot open shared object file: No > such file or directory > [28/Aug/2021:13:15:53.228547473 -0500] - ERR - symload_report_error - Could > not open library "/usr/lib64/dirsrv/plugins/libipa_cldap.so" for plugin > ipa_cldap > [28/Aug/2021:13:15:53.231054342 -0500] - ERR - load_plugin_entry - Unable to > load plugin "cn=ipa_cldap,cn=plugins,cn=config" > [28/Aug/2021:13:17:13.917125368 -0500] - NOTICE - config_set_port - > Non-Secure Port Disabled > [28/Aug/2021:13:17:13.932712979 -0500] - ERR - symload_report_error - > Netscape Portable Runtime error -5977: > /usr/lib64/dirsrv/plugins/libipa_cldap.so: cannot open shared object file: No > such file or directory > [28/Aug/2021:13:17:13.935253118 -0500] - ERR - symload_report_error - Could > not open library "/usr/lib64/dirsrv/plugins/libipa_cldap.so" for plugin > ipa_cldap > [28/Aug/2021:13:17:13.937761206 -0500] - ERR - load_plugin_entry - Unable to > load plugin "cn=ipa_cldap,cn=plugins,cn=config" > > Can anyone offer troubleshooting suggestions? Do you need a debug file or is > this log enough? Thanks in advance for your input! Knowing the distribution and version would help. This missing shared library is provided by [free]ipa-server-trust-ad, ipa-server, or something like it depending on the release. rob _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org<mailto:freeipa-users-le...@lists.fedorahosted.org> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure