On Wed, Nov 27, 2019 at 07:08:45PM +0800, Patrick Dung wrote: > Hi Fraser, > > I got one FreeIPA server that use the original jss in F31. It had problem > in upgrading. > But it resume upgrading after upgrading to a newer jss. However it still > have java/tomcat security problem described in > https://bugzilla.redhat.com/show_bug.cgi?id=1755634 > > I had another FreeIPA server (a master replica of the above server) that > could not finish upgrading with the updated jss in early November. > > Since I have very few data in the above FreeIPA domain, I replaced it by > creating a new one with CentOS. > > Thanks, > Patrick > Thanks for the updates Patrick.
> > On Wed, Nov 27, 2019 at 6:45 PM Fraser Tweedale <ftwee...@redhat.com> wrote: > > > Hi Patrick, > > > > I want to follow up with this. Did you get things working again? > > > > With the latest packages for both f30 and f31, I upgraded a FreeIPA > > installation from f30 to f31 without encountering any problems. > > > > Perhaps the jss issue caused the system to enter a poor state during > > the initial ipa-server-upgrade, such that subsequent runs of > > ipa-server-upgrade fail, even after upgrading jss. > > > > Anyone else have thoughts on this? Anyone able to reproduce with > > latest packages? > > > > Thanks, > > Fraser > > > > On Tue, Nov 05, 2019 at 03:25:13AM +0800, Patrick Dung via > > FreeIPA-users wrote: > > > The stack trace of my last mail should be hitting this bug: > > > https://bugzilla.redhat.com/show_bug.cgi?id=1755634 > > > > > > On Fri, Nov 1, 2019 at 3:20 PM Patrick Dung <patdung...@gmail.com> > > > wrote: > > > > > > > Hello all, 1) Resent, I used reply instead of reply all in the > > > > last mail. > > > > > > > > 2) I tailed the dirsrv log and perform a manual > > > > ipa-server-upgrade. I didn't found connection refused log in > > > > the dirsrv. > > > > > > > > 3) BTW, I had another ipa server that is a replica. Originally > > > > both freeipa server had upgrade problem. On the replica server, > > > > I tried to install jss-4.6.2-2.fc31.x86_64 (according to > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1766451) The > > > > ipa-server-upgrade is successfully run on the replica server. > > > > But there is problem when I access: https://replica:8443, the > > > > error message is shown in below. HTTP Status 500 – Internal > > > > Server Error ------------------------------ > > > > > > > > *Type* Exception Report > > > > > > > > *Message* org.apache.jasper.JasperException: Unable to compile > > > > class for JSP > > > > > > > > *Description* The server encountered an unexpected condition > > > > that prevented it from fulfilling the request. > > > > > > > > *Exception* > > > > > > > > org.apache.jasper.JasperException: > > > > org.apache.jasper.JasperException: Unable to compile class for > > > > JSP > > > > > > org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:604) > > > > > > org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:422) > > > > > > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385) > > > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329) > > > > javax.servlet.http.HttpServlet.service(HttpServlet.java:741) > > > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > java.lang.reflect.Method.invoke(Method.java:498) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > > > > > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314) > > > > > > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) > > > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > java.lang.reflect.Method.invoke(Method.java:498) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > > > > > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314) > > > > > > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253) > > > > > > > > *Root Cause* > > > > > > > > org.apache.jasper.JasperException: Unable to compile class for > > > > JSP > > > > > > org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:619) > > > > > > org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:399) > > > > > > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385) > > > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329) > > > > javax.servlet.http.HttpServlet.service(HttpServlet.java:741) > > > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > java.lang.reflect.Method.invoke(Method.java:498) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > > > > > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314) > > > > > > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) > > > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > java.lang.reflect.Method.invoke(Method.java:498) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > > > > > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314) > > > > > > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253) > > > > > > > > *Root Cause* > > > > > > > > java.security.AccessControlException: access denied > > > > ("java.util.PropertyPermission" > > > > "tolerateIllegalAmbiguousVarargsInvocation" "read") > > > > > > java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) > > > > > > java.security.AccessController.checkPermission(AccessController.java:886) > > > > java.lang.SecurityManager.checkPermission(SecurityManager.java:549) > > > > > > java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1294) > > > > java.lang.System.getProperty(System.java:717) > > > > > > org.eclipse.jdt.internal.compiler.impl.CompilerOptions.<init>(CompilerOptions.java:513) > > > > > > org.apache.jasper.compiler.JDTCompiler.generateClass(JDTCompiler.java:483) > > > > org.apache.jasper.compiler.Compiler.compile(Compiler.java:392) > > > > org.apache.jasper.compiler.Compiler.compile(Compiler.java:362) > > > > org.apache.jasper.compiler.Compiler.compile(Compiler.java:346) > > > > > > org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:603) > > > > > > org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:399) > > > > > > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385) > > > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329) > > > > javax.servlet.http.HttpServlet.service(HttpServlet.java:741) > > > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > java.lang.reflect.Method.invoke(Method.java:498) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > > > > > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314) > > > > > > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) > > > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > java.lang.reflect.Method.invoke(Method.java:498) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282) > > > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279) > > > > java.security.AccessController.doPrivileged(Native Method) > > > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > > > > > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314) > > > > > > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253) > > > > > > > > *Note* The full stack trace of the root cause is available in > > > > the server logs. > > > > > > > > > > > > 4) On the main server, I tried to upgrade jss and run > > > > ipa-server-upgrade but the error is still there. The error is > > > > the same as the original mail a few days ago. > > > > > > > > Thanks, > > > > > > > > Patrick > > > > > > > > On Thu, Oct 31, 2019 at 12:25 PM Patrick Dung > > > > <patdung...@gmail.com> wrote: > > > > > > > >> I tail the dirsrv log and perform a manual ipa-server-upgrade. > > > >> I didn't found connection refused log in the dirsrv. > > > >> > > > >> Thanks, Patrick > > > >> > > > >> On Thu, Oct 31, 2019 at 7:43 AM Fraser Tweedale > > > >> <ftwee...@redhat.com> wrote: > > > >> > > > >>> Is there anything in the dirsrv log relating to the connection > > > >>> attempt? Connection Refused could in fact be a TLS handshake > > > >>> error (the TLS handshake also includes certificate > > > >>> authentication). > > > >>> > > > >>> Cheers, Fraser > > > >>> > > > >>> On Wed, Oct 30, 2019 at 10:47:54PM +0800, Patrick Dung via > > > >>> FreeIPA-users wrote: > > > >>> > Hello Endi, > > > >>> > > > > >>> > The DS is up. > > > >>> > > > > >>> > $ ldapsearch -LLL -x -H ldaps://home.local.nonet:636 -b > > > >>> > "cn=users,cn=accounts,dc=local,dc=nonet" dn: > > > >>> > cn=users,cn=accounts,dc=local,dc=nonet objectClass: top > > > >>> > objectClass: nsContainer cn: users > > > >>> > > > > >>> > dn: uid=admin,cn=users,cn=accounts,dc=local,dc=nonet > > > >>> > objectClass: top objectClass: person objectClass: > > > >>> > posixaccount objectClass: krbprincipalaux objectClass: > > > >>> > krbticketpolicyaux objectClass: inetuser objectClass: > > > >>> > ipaobject objectClass: ipasshuser objectClass: > > > >>> > ipaSshGroupOfPubKeys uid: admin cn: Administrator sn: > > > >>> > Administrator uidNumber: 700000 gidNumber: 700000 > > > >>> > homeDirectory: /home/admin loginShell: /bin/bash gecos: > > > >>> > Administrator > > > >>> > > > > >>> > I had include more logs as attachment. > > > >>> > > > > >>> > Thanks, Patrick > > > >>> > > > > >>> > On Wed, Oct 30, 2019 at 10:23 PM Endi Sukma Dewata > > > >>> > <edew...@redhat.com > > > >>> > > > > >>> > wrote: > > > >>> > > > > >>> > > Hi Patrick, > > > >>> > > > > > >>> > > I see that you included the CA debug log: > > > >>> > > > > > >>> > > 2019-10-30 05:03:50 [main] FINE: LdapBoundConnection: > > > >>> > > Connecting to home.local.nonet:636 with client cert auth > > > >>> > > 2019-10-30 05:03:50 [main] FINE: > > > >>> ldapconn/PKISocketFactory.makeSSLSocket: > > > >>> > > begins 2019-10-30 05:03:50 [main] SEVERE: Unable to create > > > >>> > > socket: java.net.ConnectException: Connection refused > > > >>> > > (Connection refused) java.net.ConnectException: Connection > > > >>> > > refused (Connection refused) at > > > >>> > > java.net.PlainSocketImpl.socketConnect(Native Method) at > > > >>> > > java.net > > > >>> > > > > .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) > > > >>> > > at java.net > > > >>> > > > > > >>> > > .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) > > > >>> > > at java.net > > > >>> > > > > .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) > > > >>> > > at > > > >>> > > > > java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > > >>> > > at java.net.Socket.connect(Socket.java:607) at > > > >>> > > java.net.Socket.connect(Socket.java:556) at > > > >>> > > java.net.Socket.<init>(Socket.java:452) at > > > >>> > > java.net.Socket.<init>(Socket.java:262) > > > >>> > > > > > >>> > > The stack trace above is incomplete so it's hard to tell > > > >>> > > exactly > > > >>> where > > > >>> > > in PKI code the exception happened, but the earlier > > > >>> > > message seem to indicate that it's trying to connect to > > > >>> > > the the DS, so you need to > > > >>> make > > > >>> > > sure the DS is running and accessible. > > > >>> > > > > > >>> > > -- Endi S. Dewata > > > >>> > > > > > >>> > > ----- Original Message ----- > > > >>> > > > Hi Patrick, > > > >>> > > > > > > >>> > > > The "javax.ws.rs.ServiceUnavailableException: Subsystem > > > >>> unavailable" > > > >>> > > > suggests that the CA subsystem got undeployed > > > >>> > > > automatically from Tomcat due to an error in CA (the > > > >>> > > > Tomcat itself seems to be running just fine). > > > >>> > > > > > > >>> > > > You'll need to check the CA debug log in > > > >>> /var/log/pki/pki-tomcat/ca/*, > > > >>> > > > hopefully it will show the actual problem. Once it's > > > >>> > > > fixed, you > > > >>> just > > > >>> > > > need to restart Tomcat and the CA should be redeployed > > > >>> automatically. > > > >>> > > > > > > >>> > > > Also check the upgrade log in > > > >>> > > > /var/log/pki/pki-server-upgrade-* to > > > >>> see > > > >>> > > > if there's any PKI upgrade issue. > > > >>> > > > > > > >>> > > > Hope this helps. > > > >>> > > > > > > >>> > > > -- Endi S. Dewata > > > >>> > > > > > > >>> > > > ----- Original Message ----- > > > >>> > > > > Looks like it's the second problem (on pagure) > > > >>> > > > > > > > >>> > > > > -- Logs begin at Wed 2019-10-30 02:34:10 HKT, end at > > > >>> > > > > Wed > > > >>> 2019-10-30 > > > >>> > > > > 06:28:21 HKT. -- Oct 30 03:39:43 home.local.nonet > > > >>> > > > > systemd[1]: Starting PKI Tomcat > > > >>> Server > > > >>> > > > > pki-tomcat... Oct 30 03:39:44 home.local.nonet > > > >>> > > > > pki-server[57211]: ---------------------------- Oct 30 > > > >>> > > > > 03:39:44 home.local.nonet pki-server[57211]: > > > >>> > > > > pki-tomcat > > > >>> instance > > > >>> > > > > migrated Oct 30 03:39:44 home.local.nonet > > > >>> > > > > pki-server[57211]: ---------------------------- Oct 30 > > > >>> > > > > 03:39:44 home.local.nonet systemd[1]: Started PKI > > > >>> > > > > Tomcat > > > >>> Server > > > >>> > > > > pki-tomcat. Oct 30 03:39:44 home.local.nonet > > > >>> > > > > server[57330]: Java virtual > > > >>> machine > > > >>> > > used: > > > >>> > > > > /usr/lib/jvm/jre-1.8.0-openjdk/bin/java Oct 30 > > > >>> > > > > 03:39:44 home.local.nonet server[57330]: classpath > > > >>> > > > > used: > > > >>> > > > > > > > >>> > > > > > >>> > > /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/lib/java/commons-daemon.jar > > > >>> > > > > Oct 30 03:39:44 home.local.nonet server[57330]: main > > > >>> > > > > class used: org.apache.catalina.startup.Bootstrap Oct > > > >>> > > > > 30 03:39:44 home.local.nonet server[57330]: flags > > > >>> > > > > used: -Djava.library.path=/usr/lib64/nuxwdog-jni Oct > > > >>> > > > > 30 03:39:44 home.local.nonet server[57330]: options > > > >>> > > > > used: -Dcatalina.base=/var/lib/pki/pki-tomcat > > > >>> > > -Dcatalina.home=/usr/share/tomcat > > > >>> > > > > -Djava.endorsed.dirs= > > > >>> -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp > > > >>> > > > > > > > >>> > > > > > >>> > > -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties > > > >>> > > > > > > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > > > >>> > > > > -Djava.security.manager > > > >>> > > > > > > > >>> -Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy > > > >>> > > > > Oct 30 03:39:44 home.local.nonet server[57330]: > > > >>> > > > > arguments used: > > > >>> start > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: > > > >>> > > > > WARNING: > > > >>> Exception > > > >>> > > > > processing realm > > > >>> > > > > [com.netscape.cms.tomcat.ProxyRealm@296c31c9] > > > >>> > > > > background > > > >>> process > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: > > > >>> > > > > javax.ws.rs.ServiceUnavailableException: Subsystem > > > >>> > > > > unavailable Oct 30 03:40:03 home.local.nonet > > > >>> > > > > server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:142) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1137) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5566) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1353) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1335) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > > > > >>> > > > > > >>> > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > > > >>> > > > > Oct 30 03:40:03 home.local.nonet server[57330]: at > > > >>> > > > > java.lang.Thread.run(Thread.java:748) > > > >>> > > > > > > > >>> > > > > The pki-tomcat instance is running. It would output > > > >>> > > > > error when I > > > >>> browse > > > >>> > > > > https://home.local.nonet:8443/ca , but > > > >>> > > https://home.local.nonet:8443/pki/ > > > >>> > > > > is ok Instance ID: pki-tomcat Active: True Unsecure > > > >>> > > > > Port: 8080 Secure Port: 8443 AJP Port: 8009 Tomcat > > > >>> > > > > Port: 8005 > > > >>> > > > > > > > >>> > > > > CA Subsystem: Type: Subordinate CA (Security Domain) > > > >>> > > > > SD Registration URL: https://home.local.nonet:443 > > > >>> > > > > Enabled: True Unsecure URL: > > > >>> > > > > http://home.local.nonet:8080/ca/ee/ca Secure Agent > > > >>> > > > > URL: https://home.local.nonet:8443/ca/agent/ca Secure > > > >>> > > > > EE URL: https://home.local.nonet:8443/ca/ee/ca Secure > > > >>> > > > > Admin URL: https://home.local.nonet:8443/ca/services > > > >>> > > > > PKI Console URL: https://home.local.nonet:8443/ca > > > >>> > > > > > > > >>> > > > > Thanks, Patrick > > > >>> > > > > > > > >>> > > > > On Wed, Oct 30, 2019 at 5:44 AM Alex Scheel < > > > >>> > > > > asch...@redhat.com > > > >>> > > > > >>> > > wrote: > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > You might try checking journalctl output. > > > >>> > > > > > > > >>> > > > > It might be this bug: > > > >>> > > https://bugzilla.redhat.com/show_bug.cgi?id=1766451 > > > >>> > > > > > > > >>> > > > > Otherwise, it is a perfect match for this bug: > > > >>> > > > > https://pagure.io/dogtagpki/issue/3111 > > > >>> > > > > > > > >>> > > > > Which I'd also like journalctl output on, if you have > > > >>> > > > > any to > > > >>> share. :) > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > I should have a Bodhi update out tonight yet for the > > > >>> > > > > issue in > > > >>> the BZ. > > > >>> > > > > Without more information, I'm not sure we'd know what > > > >>> > > > > cause for the > > > >>> second > > > >>> > > issue > > > >>> > > > > is. > > > >>> > > > > > > > >>> > > > > - Alex > > > >>> > > > > > > > >>> > > > > ----- Original Message ----- > > > >>> > > > > > From: "Patrick Dung via FreeIPA-users" < > > > >>> > > > > > freeipa-users@lists.fedorahosted.org > To: > > > >>> > > > > > freeipa-users@lists.fedorahosted.org Cc: "Patrick > > > >>> > > > > > Dung" < patdung...@gmail.com > Sent: Tuesday, > > > >>> > > > > > October 29, 2019 5:29:09 PM Subject: [Freeipa-users] > > > >>> > > > > > FreeIPA having problem after > > > >>> upgrading from > > > >>> > > > > > Fedora 30 to 31 > > > >>> > > > > > > > > >>> > > > > > Hello, > > > >>> > > > > > > > > >>> > > > > > I got problem upgrading from FC30 to FC31. Before > > > >>> > > > > > upgrade the FreeIPA in FC30 is running fine. > > > >>> > > > > > > > > >>> > > > > > After OS upgrade, IPA cannot start and checked that > > > >>> > > > > > it stuck > > > >>> at the > > > >>> > > CA > > > >>> > > > > > part. I run ipa-server-upgrade manually but there > > > >>> > > > > > is problem. > > > >>> > > > > > > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG request GET > > > >>> > > > > > https://home.local.nonet:8443/ca/rest/account/login > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG request body '' > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG response status 500 > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG response headers > > > >>> > > > > > Content-Type: text/html;charset=utf-8 > > > >>> > > > > > Content-Language: en Content-Length: 2481 Date: Tue, > > > >>> > > > > > 29 Oct 2019 21:03:58 GMT Connection: close > > > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG response body (decoded): > > > >>> > > > > > b'<!doctype html><html lang="en"><head><title>HTTP > > > >>> > > > > > Status 500 \xe2\x80\x93 Internal > > > >>> Server > > > >>> > > > > > Error</title><style type="text/css">h1 > > > >>> > > > > > > > > >>> > > > > > >>> > > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} > > > >>> > > > > > h2 > > > >>> > > > > > > > > >>> > > > > > >>> > > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} > > > >>> > > > > > h3 > > > >>> > > > > > > > > >>> > > > > > >>> > > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} > > > >>> > > > > > body > > > >>> > > > > > > > > >>> > > > > > >>> > > {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} > > > >>> > > > > > b > > > >>> > > > > > > > > >>> > > > > > >>> > > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} > > > >>> > > > > > p > > > >>> > > > > > > > > >>> > > > > > >>> > > {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;} > > > >>> > > > > > a {color:black;} a.name {color:black;} .line > > > >>> > > > > > > > > >>> > > > > > >>> > > {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP > > > >>> > > > > > Status 500 \xe2\x80\x93 Internal Server > > > >>> > > > > > Error</h1><hr > > > >>> class="line" > > > >>> > > > > > /><p><b>Type</b> Exception > > > >>> > > > > > Report</p><p><b>Message</b> > > > >>> Subsystem > > > >>> > > > > > unavailable</p><p><b>Description</b> The server > > > >>> > > > > > encountered an > > > >>> > > unexpected > > > >>> > > > > > condition that prevented it from fulfilling the > > > >>> > > > > > request.</p><p><b>Exception</b></p><pre> javax.ws.rs > > > >>> > > > > > .ServiceUnavailableException: Subsystem > > > >>> > > > > > > > > >>> > > > > > >>> > > unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:150)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:515)\n\tcom.netscape.cms.tomcat.ExternalAuthenticationValve.invoke(ExternalAuthenticationValve.java:82)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)\n\torg.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)\n\torg.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)\n\torg.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)\n\ > > > >>> > > > > > torg.apache.tomcat.util.net > > > >>> > > > > > > > .NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)\n\ > > > >>> > > > > > torg.apache.tomcat.util.net > > > >>> > > > > > > > > >>> > > > > > >>> > > .SocketProcessorBase.run(SocketProcessorBase.java:49)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>Note</b> > > > >>> > > > > > The full stack trace of the root cause is available > > > >>> > > > > > in the > > > >>> server > > > >>> > > > > > logs.</p><hr class="line" /><h3>Apache > > > >>> > > Tomcat/9.0.26</h3></body></html>' > > > >>> > > > > > 2019-10-29T21:03:58Z ERROR IPA server upgrade > > > >>> > > > > > failed: Inspect /var/log/ipaupgrade.log and run > > > >>> > > > > > command ipa-server-upgrade > > > >>> manually. > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG File > > > >>> > > > > > "/usr/lib/python3.7/site-packages/ipapython/admintool.py", > > > >>> line 179, > > > >>> > > in > > > >>> > > > > > execute return_value = self.run() File > > > >>> > > > > > > > > >>> > > > > > >>> > > "/usr/lib/python3.7/site-packages/ipaserver/install/ipa_server_upgrade.py", > > > >>> > > > > > line 54, in run server.upgrade() File > > > >>> > > > > > > > > >>> > > > > > >>> > > "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py", > > > >>> > > > > > line 2223, in upgrade upgrade_configuration() File > > > >>> > > > > > > > > >>> > > > > > >>> > > "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py", > > > >>> > > > > > line 2093, in upgrade_configuration > > > >>> > > > > > ca_enable_ldap_profile_subsystem(ca) File > > > >>> > > > > > > > > >>> > > > > > >>> > > "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py", > > > >>> > > > > > line 414, in ca_enable_ldap_profile_subsystem > > > >>> > > > > > cainstance.migrate_profiles_to_ldap() File > > > >>> > > > > "/usr/lib/python3.7/site-packages/ipaserver/install/cainstance.py", > > > >>> > > > > > line 1937, in migrate_profiles_to_ldap > > > >>> > > > > > _create_dogtag_profile(profile_id, profile_data, > > > >>> overwrite=False) > > > >>> > > > > > File > > > >>> > > > > "/usr/lib/python3.7/site-packages/ipaserver/install/cainstance.py", > > > >>> > > > > > line 1943, in _create_dogtag_profile with > > > >>> > > > > > api.Backend.ra_certprofile as profile_api: File > > > >>> "/usr/lib/python3.7/site-packages/ipaserver/plugins/dogtag.py", > > > >>> > > line > > > >>> > > > > > 1315, in __enter__ raise > > > >>> > > > > > errors.RemoteRetrieveError(reason=_('Failed to > > > >>> authenticate to > > > >>> > > CA > > > >>> > > > > > REST API')) > > > >>> > > > > > > > > >>> > > > > > 2019-10-29T21:03:58Z DEBUG The ipa-server-upgrade > > > >>> > > > > > command > > > >>> failed, > > > >>> > > > > > exception: RemoteRetrieveError: Failed to > > > >>> > > > > > authenticate to CA > > > >>> REST API > > > >>> > > > > > 2019-10-29T21:03:58Z ERROR Unexpected error - see > > > >>> > > /var/log/ipaupgrade.log > > > >>> > > > > > for details: RemoteRetrieveError: Failed to > > > >>> > > > > > authenticate to CA REST API 2019-10-29T21:03:58Z > > > >>> > > > > > ERROR The ipa-server-upgrade command > > > >>> failed. See > > > >>> > > > > > /var/log/ipaupgrade.log for more information > > > >>> > > > > > > > > >>> > > > > > From /var/log/pki/pki-tomcat/ca/debug log file: > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: LdapAuthInfo: > > > >>> > > > > > init() 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapAuthInfo: init begins 2019-10-30 05:03:50 [main] > > > >>> > > > > > FINEST: Getting > > > >>> > > > > > internaldb.ldapauth.authtype=SslClientAuth > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: LdapAuthInfo: init > > > >>> > > > > > ends 2019-10-30 05:03:50 [main] FINEST: Property > > > >>> internaldb.errorIfDown > > > >>> > > not > > > >>> > > > > > found 2019-10-30 05:03:50 [main] FINEST: Getting > > > >>> > > internaldb.errorIfDown=true > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST: Property > > > >>> internaldb.doCloning not > > > >>> > > > > > found 2019-10-30 05:03:50 [main] FINEST: Getting > > > >>> internaldb.doCloning=true > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: > > > >>> doCloning: > > > >>> > > true > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: > > > >>> mininum: 3 > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: > > > >>> maximum: 15 > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: host: home.local.nonet > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: port: > > > >>> 636 > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: secure: > > > >>> true > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: > > > >>> > > authentication: 2 > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > LdapBoundConnFactory: makeConnection(true) > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST: Getting > > > >>> > > > > > internaldb.ldapauth.clientCertNickname=subsystemCert > > > >>> cert-pki-ca > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST: Property > > > >>> > > > > > tcp.keepAlive not > > > >>> found > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST: Getting > > > >>> > > > > > tcp.keepAlive=true 2019-10-30 05:03:50 [main] FINE: > > > >>> > > > > > TCP Keep-Alive: true 2019-10-30 05:03:50 [main] > > > >>> > > > > > FINE: LdapBoundConnection: > > > >>> Connecting to > > > >>> > > > > > home.local.nonet:636 with client cert auth > > > >>> > > > > > 2019-10-30 05:03:50 [main] FINE: > > > >>> > > ldapconn/PKISocketFactory.makeSSLSocket: > > > >>> > > > > > begins 2019-10-30 05:03:50 [main] SEVERE: Unable to > > > >>> > > > > > create socket: java.net.ConnectException: Connection > > > >>> > > > > > refused (Connection > > > >>> refused) > > > >>> > > > > > java.net.ConnectException: Connection refused > > > >>> > > > > > (Connection > > > >>> refused) > > > >>> > > > > > at java.net.PlainSocketImpl.socketConnect(Native > > > >>> > > > > > Method) at java.net > > > >>> > > > > > > > > >>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) > > > >>> > > > > > at java.net > > > >>> > > > > > > > > >>> > > > > > >>> > > .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) > > > >>> > > > > > at java.net > > > >>> > > > > > > > > >>> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) > > > >>> > > > > > at > > > >>> > > > > > java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > > >>> > > > > > at java.net.Socket.connect(Socket.java:607) at > > > >>> > > > > > java.net.Socket.connect(Socket.java:556) at > > > >>> > > > > > java.net.Socket.<init>(Socket.java:452) at > > > >>> > > > > > java.net.Socket.<init>(Socket.java:262) > > > >>> > > > > > > > > >>> > > > > > Some error is logged to /var/log/messages: Oct 30 > > > >>> > > > > > 05:26:50 home server[65722]: WARNING: Exception > > > >>> processing > > > >>> > > realm > > > >>> > > > > > [com.netscape.cms.tomcat.ProxyRealm@5647a92b] > > > >>> > > > > > background > > > >>> process > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: javax.ws.rs > > > >>> > > > > > .ServiceUnavailableException: Subsystem > > > >>> unavailable > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:142) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1137) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5566) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1353) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1335) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > > > > >>> > > > > > >>> > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > > > >>> > > > > > Oct 30 05:26:50 home server[65722]: #011at > > > >>> > > > > > java.lang.Thread.run(Thread.java:748) > > > >>> > > > > > > > > >>> > > > > > I am able to connect to my ldap server port 636 with > > > >>> > > > > > TLS > > > >>> without > > > >>> > > problem. > > > >>> > > > > > > > > >>> > > > > > Thanks, Patrick > > > >>> > > > > > > > > >>> > > > > > _______________________________________________ > > > >>> > > > > > FreeIPA-users mailing list -- > > > >>> freeipa-users@lists.fedorahosted.org > > > >>> > > > > > To unsubscribe send an email to > > > >>> > > > > > freeipa-users-le...@lists.fedorahosted.org Fedora > > > >>> > > > > > Code of Conduct: > > > >>> > > > > > > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > > >>> > > > > > List Guidelines: > > > >>> > > https://fedoraproject.org/wiki/Mailing_list_guidelines > > > >>> > > > > > List Archives: > > > >>> > > > > > > > > >>> > > > > > >>> > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > _______________________________________________ > > > >>> > > > > FreeIPA-users mailing list -- > > > >>> freeipa-users@lists.fedorahosted.org > > > >>> > > > > To unsubscribe send an email to > > > >>> > > freeipa-users-le...@lists.fedorahosted.org > > > >>> > > > > Fedora Code of Conduct: > > > >>> > > > > > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > > >>> > > > > List Guidelines: > > > >>> > > https://fedoraproject.org/wiki/Mailing_list_guidelines > > > >>> > > > > List Archives: > > > >>> > > > > > > > >>> > > > > > >>> > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > >>> > > > > > > > >>> > > > _______________________________________________ > > > >>> > > > FreeIPA-users mailing list -- > > > >>> > > > freeipa-users@lists.fedorahosted.org To unsubscribe send > > > >>> > > > an email to > > > >>> > > freeipa-users-le...@lists.fedorahosted.org > > > >>> > > > Fedora Code of Conduct: > > > >>> > > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > > >>> > > > List Guidelines: > > > >>> https://fedoraproject.org/wiki/Mailing_list_guidelines > > > >>> > > > List Archives: > > > >>> > > > > > > >>> > > > > > >>> > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > >>> > > > > > > >>> > > > > > >>> > > > > > >>> > > > >>> > > > >>> > _______________________________________________ > > > >>> > FreeIPA-users mailing list -- > > > >>> > freeipa-users@lists.fedorahosted.org To unsubscribe send an > > > >>> > email to > > > >>> freeipa-users-le...@lists.fedorahosted.org > > > >>> > Fedora Code of Conduct: > > > >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > > >>> > List Guidelines: > > > >>> https://fedoraproject.org/wiki/Mailing_list_guidelines > > > >>> > List Archives: > > > >>> > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > >>> > > > >>> > > > > > _______________________________________________ > > > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > > > To unsubscribe send an email to > > freeipa-users-le...@lists.fedorahosted.org > > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > > List Archives: > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > > _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
