Mark,
Thanks for continuing to look into it, and producing a detailed record of
the issues and the cause (along with intermediate details). Hopefully it
will come in useful for others in the future.
Robert
On Thu, Mar 3, 2022 at 3:11 AM Mark Thomas <ma...@apache.org> wrote:
> Robert,
>
> Apologies for the delayed reply. Having found the root cause of the
> issue I wanted to confirm whether or not the Docker desktop team viewed
> it as a security vulnerability or not. I received confirmation yesterday
> that they do not.
>
> TL;DR
> Running Tomcat in a container via Docker Desktop on a Windows host with
> the web application served from a location on the host mounted/bound to
> the container is insecure.
>
> Longer version:
> To ensure that things like security constraints are correctly applied,
> Tomcat has to process URLs in a case sensitive manner. "/Test.jsp" is
> not he same as "/test.jsp".
>
> URLs are often mapped to resources on the file system. This mapping also
> needs to be applied in a case sensitive manner. If the file system is
> case sensitive, this is relatively straight forward. If the file system
> is case insensitive, Tomcat performs some additional checks. These look
> something like this:
> - Request for "/Test.jsp"
> - Find file "$appbase/Test.jsp"
> - Get the canonical path
> - Confirm the case matches the original request
>
> On a case insensitive file system, looking up "$appbase/Test.jsp" will
> match "$appbase/test.jsp" but the canonical path will return
> "$appbase/test.jsp" which doesn't match so a 404 will be returned.
>
> The issue with Docker Desktop is that paths on the Windows host
> mounted/bound to the container behave like this:
> - The path - as far as Windows is concerned - is "$appbase/Test.jsp"
> - Code running in the container looks up "$appbase/Test.jsp"
> - The file is found
> - The canonical path is "$appbase/Test.jsp"
> - So far, so good
> - Code running in the container looks up "$appbase/test.jsp"
> - The file is found
> - The canonical path is "$appbase/test.jsp"
>
> The issue is that the canonical path returned the second time matches
> the path used to obtain the resource, not the canonical path on the
> Windows file system. This means Tomcat cannot perform the case
> sensitivity tests.
> This creates the following security issues:
> - security constraints can be bypasses
> - JSP source code disclosure
> It also creates class loading issues as Java class and package names are
> also case sensitive.
>
> The view of the Docket Desktop team is that this is, at best, a bug not
> a security vulnerability because Docker Desktop is a developer tool, not
> a tool for running production instances. Further, the expectation is
> that the web application would be included in the container in production.
>
> If you do continue to use this approach in development, keep in mind that:
> - you may see issues like the original EL issue you reported
> - security testing may report false positives
>
> HTH,
>
> Mark
>
>
>
>
> On 08/02/2022 15:11, Robert Turner wrote:
> > Okay. Yep, my most recent suspicion was correct -- it's related to the
> > Docker bind to a local folder containing the webapps. As such, I believe
> > it's a Docker issue of some sort and not Tomcat specific. However, you
> may
> > want to understand it more completely in any case.
> >
> > Thanks for your help Mark, Rob S and Neil.
> >
> >
> > Here are the full details of the reproduction scenario:
> >
> > Host system: MacOS 12.2; Docker Desktop v4.4.2 (73305), Engine 20.10.12
> >
> > 1. Using Maven Archetypes, do the following:
> >
> > mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes
> > -DarchetypeArtifactId=maven-archetype-webapp -DarchetypeVersion=1.4
> >
> > When prompted for configuration values, use the following:
> >
> > groupId: com.example.rt
> > artifactId: test-el-resolver
> > version: 1.0-SNAPSHOT
> > package: com.example.rt
> >
> > 2. Switch to the new folder created for the web app
> >
> > cd test-el-resolver
> >
> > 3. Modify the index.jsp file (src/main/webapp/index.jsp) to have the
> > following contents:
> >
> > <%@page contentType="text/html" pageEncoding="UTF-8" isELIgnored="false"
> %>
> > <%@page import="com.example.rt.Failing"%>
> > <%
> > final Failing failing = null;
> > pageContext.setAttribute("failing", failing);
> > %>
> > <html>
> > <body>
> > <h2>Hello World!</h2>
> >
> > <p>field1=${failing.field1}</p>
> > </body>
> > </html>
> >
> > 4. Add the following contents to
> > file src/main/java/com/example/rt/Failing.java:
> >
> > package com.example.rt;
> >
> > public class Failing {
> >
> > private final String field1 = "field1_value";
> >
> > public String getField1() {
> > return field1;
> > }
> >
> > }
> >
> >
> > 5. Build the web app
> >
> > mvn package
> >
> >
> > 6. Create a local folder to mount/bind into the Docker container
> >
> > mkdir webapps
> >
> >
> > 7. Copy the WAR file to the new folder
> >
> > cp target/test-el-resolver.war webapps/
> >
> >
> > 8. Start a docker container binding the local web apps folder
> >
> > docker run -d -p 8075:8080 --mount
> > type=volume,src=`pwd`/webapps,dst=/usr/local/tomcat/webapps
> > tomcat:9.0.58-jre11-openjdk
> >
> >
> > 9. Using a browser (or equivalent), access the app:
> >
> > curl http://localhost:8075/test-el-resolver/
> >
> >
> > 10. Observe the following exception in the error page:
> >
> > Stacktrace:
> >
> org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:610)
> >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:489)
> > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
> > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
> > javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> > </pre><p><b>Root Cause</b></p><pre>javax.servlet.ServletException:
> > java.lang.NoClassDefFoundError: com/example/rt/Failing (wrong
> > name: com/example/rt/failing)
> >
> org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:657)
> > org.apache.jsp.index_jsp._jspService(index_jsp.java:145)
> > org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
> > javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:466)
> > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
> > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
> > javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> > </pre><p><b>Root Cause</b></p><pre>java.lang.NoClassDefFoundError:
> > com/example/rt/Failing (wrong name:
> > com/example/rt/failing)
> > java.base/java.lang.ClassLoader.defineClass1(Native Method)
> > java.base/java.lang.ClassLoader.defineClass(Unknown Source)
> > java.base/java.security.SecureClassLoader.defineClass(Unknown Source)
> >
> org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2478)
> >
> org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:870)
> >
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1371)
> >
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1215)
> > javax.el.ImportHandler.findClass(ImportHandler.java:477)
> > javax.el.ImportHandler.resolveClass(ImportHandler.java:421)
> >
> javax.servlet.jsp.el.ScopedAttributeELResolver.getValue(ScopedAttributeELResolver.java:85)
> > org.apache.jasper.el.JasperELResolver.getValue(JasperELResolver.java:124)
> > org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:93)
> > org.apache.el.parser.AstValue.getValue(AstValue.java:136)
> > org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:189)
> >
> org.apache.jasper.runtime.PageContextImpl.proprietaryEvaluate(PageContextImpl.java:692)
> > org.apache.jsp.index_jsp._jspService(index_jsp.java:130)
> > org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
> > javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:466)
> > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
> > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
> > javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> > </pre><p><b>Note</b> The full stack trace of the root cause is available
> in
> > the server logs.</p><hr class="line" /><h3>Apache
> > Tomcat/9.0.58</h3></body></html>
> >
> >
> > And a "working" scenario, replace step 8 with the following:
> >
> > docker run -d -p 8075:8080 tomcat:9.0.58-jre11-openjdk
> > docker cp target/test-e-resolver.war
> > <container_id>:/usr/local/tomcat/webapps/
> >
> > Accessing the page with the curl command above will result in the expect
> > page being displayed (and no exception).
> >
> >
> > Robert
> >
> >
> >
> > On Tue, Feb 8, 2022 at 9:39 AM Robert Turner <rtur...@e-djuster.ca>
> wrote:
> >
> >> Mark,
> >>
> >> Thanks for the quick follow up.
> >>
> >> Based on your comments I have some ideas on what it might be, and I
> should
> >> be able to narrow that down further for you, and provide all the details
> >> that you requested.
> >>
> >> A few notes on the environment I'm using:
> >>
> >> Docker (the host) is running on my Mac (12.2), not in AWS. AWS Linux is
> >> only involved in the Docker image (as per the Dockerfile I shared
> earlier).
> >> Moreover, it happens with Debian in the Docker image as well.
> >> What I think is important is that I have "<tomcat>/work" and
> >> "<tomcat>/webapps" mounted to file folders on the Mac, so I am
> suspecting
> >> it's something to do with that "mounting" of the volume to the host OS,
> and
> >> how Docker is mapping those and handling filename casing, etc. My MacOS
> >> file system is APFS, Encrypted (and I thought I had case sensitivity
> >> enabled, but I can no longer see that option -- maybe not an option for
> >> APFS).
> >>
> >> I will try to confirm suspicions and provide details in a few hours
> >> (hopefully -- got a few meetings today that will get in the way).
> >>
> >> Thanks again,
> >>
> >> Robert
> >>
> >>
> >> On Tue, Feb 8, 2022 at 8:51 AM Mark Thomas <ma...@apache.org> wrote:
> >>
> >>> Robert,
> >>>
> >>> I agree this is something to do with the Docker environment.
> >>>
> >>> I think case insensitivity is involved somewhere as I can trigger the
> >>> error if I copy my equivalent of Failure.class to failure.class and
> then
> >>> call the JSP.
> >>>
> >>> I understand why it only occurs for * imports. In that instance, Tomcat
> >>> has to check if the class can be loaded from each of the imported
> >>> packages. Tomcat checks the file system first as that is faster than
> >>> trying (and failing) to load a class. The file system checks are
> >>> designed to be case sensitive - even on case insensitive file systems.
> >>> Something seems to be going wrong here and I'm still not sure what.
> >>>
> >>> I have tried to recreate this on AWS without success. If you have time,
> >>> I think we'd need the following to dig into this further:
> >>>
> >>> - Source for the simplest possible test WAR that demonstrates this
> >>> issue. I think a single class and a single JSP page should be
> >>> sufficient.
> >>>
> >>> - The WAR you created from the above (to rule out any build issues).
> >>>
> >>> - A minimal Dockerfile to create a Tomcat instance that demonstrates
> >>> this issue. Should just copy the WAR to the container and start it
> >>> with JPDA enabled.
> >>>
> >>> - Which AMI you used to create the AWS instance. I'm using the AWS free
> >>> tier so I used a t2.micro instance with
> >>> amzn2-ami-kernel-5.10-hvm-2.0.20220121.0-x86_64-gp2
> >>>
> >>> Thanks,
> >>>
> >>> Mark
> >>>
> >>>
> >>> On 08/02/2022 13:24, Robert Turner wrote:
> >>>> One thing to add is that my "conclusion" about OS variances I believe
> >>> to be
> >>>> incorrect. Our tests typically run on Linux, so I think it's still
> >>>> something to do with a difference in the Docker-based environment.
> >>>>
> >>>> Let me know if you need any more details on anything...(but I suspect
> >>> with
> >>>> a debugger up on the Expression Resolvers, you will at least narrow it
> >>> down
> >>>> quickly...)
> >>>>
> >>>> On Tue, Feb 8, 2022 at 7:55 AM Robert Turner <rtur...@e-djuster.ca>
> >>> wrote:
> >>>>
> >>>>> Thanks Mark. Much appreciated.
> >>>>>
> >>>>> On Tue., Feb. 8, 2022, 04:06 Mark Thomas, <ma...@apache.org> wrote:
> >>>>>
> >>>>>> Robert,
> >>>>>>
> >>>>>> Thank you for putting the effort in to debugging this. Narrowing
> down
> >>>>>> the issue to a simple test case is extremely helpful.
> >>>>>>
> >>>>>> The behaviour you describe looks odd to me. I'd expect consistent
> >>>>>> behaviour across platforms irrespective of the case sensitivity of
> the
> >>>>>> file system in use.
> >>>>>>
> >>>>>> I'm going to use your test case to investigate this further. I'll
> >>> report
> >>>>>> back here with my findings - hopefully later today.
> >>>>>>
> >>>>>> Mark
> >>>>>>
> >>>>>>
> >>>>>> On 08/02/2022 03:29, Robert Turner wrote:
> >>>>>>> Okay, so I have finally narrowed it down the trivial failure case,
> >>> and I
> >>>>>>> think I have an explanation as a result:
> >>>>>>>
> >>>>>>> [1] works (in docker), and [2] fails (in docker) but works outside.
> >>> The
> >>>>>>> difference between the two is the import directive being a wildcard
> >>>>>> (ugly,
> >>>>>>> but historical in our app in some places we haven't yet cleaned
> up).
> >>>>>>>
> >>>>>>> I am therefore speculating based on the Expression Language
> >>>>>> specification
> >>>>>>> that because the class wasn't explicitly imported, it's not in the
> >>> list
> >>>>>> of
> >>>>>>> available classes for static class resolution, and thus it fails.
> >>>>>>> Combine this with MacOS and Windows not caring about filename
> cases,
> >>> and
> >>>>>>> Linux caring, then I suspect it's just matching differently in both
> >>>>>> cases.
> >>>>>>>
> >>>>>>> Workaround/fix would be:
> >>>>>>> - to ensure we explicitly import the class (instead or in addition
> to
> >>>>>> the
> >>>>>>> wildcard)
> >>>>>>> OR
> >>>>>>> - rename the attribute so it doesn't map directly to the class
> name.
> >>>>>>>
> >>>>>>>
> >>>>>>> So I think I can bring my overly-detailed thread to an end...unless
> >>> my
> >>>>>>> guess at the reasoning is incorrect and someone has a better
> >>>>>> explanation.
> >>>>>>>
> >>>>>>> Thanks Rob S and Neil for having a look and providing suggestions
> --
> >>> in
> >>>>>>> part, it was something related to what you both said, but I believe
> >>> in
> >>>>>>> different contexts than you expected.
> >>>>>>>
> >>>>>>> Robert
> >>>>>>>
> >>>>>>>
> >>>>>>> [1]
> >>>>>>> $ cat src/main/webapp/index.jsp
> >>>>>>> <%@page contentType="text/html" pageEncoding="UTF-8"
> >>>>>> isELIgnored="false" %>
> >>>>>>> <%@page import="com.example.rt.Failing" %>
> >>>>>>> <%
> >>>>>>> final Failing failing = null;
> >>>>>>> pageContext.setAttribute("failing", failing);
> >>>>>>> %>
> >>>>>>> <html>
> >>>>>>> <body>
> >>>>>>> <h2>Hello World!</h2>
> >>>>>>>
> >>>>>>> <p>field1=${failing.field1}</p>
> >>>>>>> </body>
> >>>>>>>
> >>>>>>> [2]
> >>>>>>> $ cat src/main/webapp/index.jsp
> >>>>>>> <%@page contentType="text/html" pageEncoding="UTF-8"
> >>>>>> isELIgnored="false" %>
> >>>>>>> <%@page import="com.example.rt.*" %>
> >>>>>>> <%
> >>>>>>> final Failing failing = null;
> >>>>>>> pageContext.setAttribute("failing", failing);
> >>>>>>> %>
> >>>>>>> <html>
> >>>>>>> <body>
> >>>>>>> <h2>Hello World!</h2>
> >>>>>>>
> >>>>>>> <p>field1=${failing.field1}</p>
> >>>>>>> </body>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Feb 7, 2022 at 10:14 PM Robert Turner <
> rtur...@e-djuster.ca>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> So back to a divide and conquer approach I think.
> >>>>>>>>
> >>>>>>>> I just created a trivial "example" [1] and it works as expected
> >>> (i.e.
> >>>>>> no
> >>>>>>>> exception was generated) (on the same servers I was testing the
> >>>>>> complex JAR
> >>>>>>>> file) -- so possibly something else modifying the behaviour -- a
> JAR
> >>>>>> on the
> >>>>>>>> classpath, or something in the JSP file...
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>>>> $ cat src/main/webapp/index.jsp
> >>>>>>>> <%@page contentType="text/html" pageEncoding="UTF-8"
> >>>>>> isELIgnored="false" %>
> >>>>>>>> <%@page import="com.example.rt.Failing"%>
> >>>>>>>> <%
> >>>>>>>> final Failing failing = null;
> >>>>>>>> pageContext.setAttribute("failing", failing);
> >>>>>>>> %>
> >>>>>>>> <html>
> >>>>>>>> <body>
> >>>>>>>> <h2>Hello World!</h2>
> >>>>>>>>
> >>>>>>>> <p>field1=${failing.field1}</p>
> >>>>>>>> </body>
> >>>>>>>> </html>
> >>>>>>>>
> >>>>>>>> $ cat src/main/java/com/example/rt/Failing.java
> >>>>>>>> package com.example.rt;
> >>>>>>>>
> >>>>>>>> public class Failing {
> >>>>>>>>
> >>>>>>>> private final String field1 = "field1_value";
> >>>>>>>>
> >>>>>>>> public String getField1() {
> >>>>>>>> return field1;
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Feb 7, 2022 at 9:51 PM Robert Turner <
> rtur...@e-djuster.ca>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I'm just avoiding sharing product details or things I think only
> >>>>>> serves
> >>>>>>>>> to confuse the problem. Sorry if you felt I wasn't sharing. It
> >>> wasn't
> >>>>>> my
> >>>>>>>>> intention to be obtuse. I didn't believe they added any value for
> >>> the
> >>>>>>>>> diagnostics (of course that assumes I know enough about the
> >>> problem).
> >>>>>>>>>
> >>>>>>>>> However, since you think they might be useful, here they the name
> >>>>>> mapping
> >>>>>>>>> from the exception and the Java and JSP code excerpts:
> >>>>>>>>>
> >>>>>>>>> "package1" -> "model"
> >>>>>>>>> "Class1" -> "Organization"
> >>>>>>>>> "class1" -> "organization"
> >>>>>>>>>
> >>>>>>>>> The class is present in the package (see [1]) -- otherwise it
> >>> wouldn't
> >>>>>>>>> work in one environment and not the other -- I believe I have
> >>>>>> confirmed
> >>>>>>>>> that carefully. Full class paths (with path-names relativised to
> >>>>>> compare
> >>>>>>>>> between environments) listed below [2], and the code that emitted
> >>> the
> >>>>>>>>> listing is here [3]. I post-processed the log lines as follows
> [4].
> >>>>>> Docker
> >>>>>>>>> file for the Tomcat container provided [5]. JDK details listed in
> >>> [6].
> >>>>>>>>> Tomcat version is 9.0.58 in all cases.
> >>>>>>>>>
> >>>>>>>>> What seems to be different is:
> >>>>>>>>> - the way the EL resolver is working
> >>>>>>>>> OR
> >>>>>>>>> - the behaviour of the class loader differs in the different
> >>>>>>>>> environments.
> >>>>>>>>> OR
> >>>>>>>>> - something else I do not understand is relevant
> >>>>>>>>>
> >>>>>>>>> The working environments are MacOS and Windows, and the failing
> >>>>>>>>> environment is either Debian or AWS Linux 2 running in a docker
> >>>>>> container.
> >>>>>>>>> If the class loaders behaviour differently, then that could
> explain
> >>>>>> the
> >>>>>>>>> issues, however, that would surprise me if they differed in any
> >>>>>> material
> >>>>>>>>> way on the different platforms.
> >>>>>>>>>
> >>>>>>>>> I hope that helps provide more detail that might be useful...
> >>>>>>>>>
> >>>>>>>>> Robert
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>> $ unzip -l target/app.war | grep "model\/Organization\.class"
> >>>>>>>>> 66246 02-07-2022 20:17
> >>>>>> WEB-INF/classes/model/Organization.class
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> [2]
> >>>>>>>>> java.class.path=/bin/bootstrap.jar:/bin/tomcat-juli.jar
> >>>>>>>>> Class loader URLs:
> >>>>>>>>> Class loader
> >>>>>> name=org.apache.catalina.loader.ParallelWebappClassLoader,
> >>>>>>>>> URL count=146
> >>>>>>>>> URL=file:/WEB-INF/classes/
> >>>>>>>>> URL=file:/WEB-INF/lib/FastInfoset-1.2.18.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/SparseBitSet-1.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/VeracodeAnnotations-1.2.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/activation-1.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/animal-sniffer-annotations-1.20.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/annotations-4.1.1.4.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/api-common-2.0.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/auto-value-annotations-1.8.2.jar
> >>>>>>>>>
> >>> URL=file:/WEB-INF/lib/avatax-rest-v2-api-java_2.11-21.12.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/aws-java-sdk-core-1.12.145.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/aws-java-sdk-kms-1.12.145.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/aws-java-sdk-s3-1.12.145.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/bcmail-jdk15on-1.70.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/bcpkix-jdk15on-1.70.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/bcprov-jdk15on-1.70.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/bcutil-jdk15on-1.70.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/castor-core-1.4.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/castor-xml-1.4.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/checker-qual-3.5.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-beanutils-1.9.4.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-codec-1.15.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-collections-3.2.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-collections4-4.4.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-compress-1.21.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-digester-2.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-fileupload-1.4.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-io-2.11.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-lang3-3.12.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-logging-1.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-math3-3.6.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/commons-text-1.9.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/conscrypt-openjdk-uber-2.5.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/curvesapi-1.06.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/ecj-3.21.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/encoder-1.2.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/encoder-jsp-1.2.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/error_prone_annotations-2.9.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/failureaccess-1.0.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/fontbox-2.0.25.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/gax-2.3.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/gax-grpc-2.3.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/gmbal-api-only-4.0.3.jar
> >>>>>>>>>
> >>> URL=file:/WEB-INF/lib/google-auth-library-credentials-1.1.0.jar
> >>>>>>>>>
> >>> URL=file:/WEB-INF/lib/google-auth-library-oauth2-http-1.1.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/google-cloud-speech-1.30.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/google-http-client-1.39.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/google-http-client-gson-1.39.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/googleauth-1.5.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-alts-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-api-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-auth-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-context-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-core-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-grpclb-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-netty-shaded-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-protobuf-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-protobuf-lite-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/grpc-stub-1.40.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/gson-2.8.9.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/guava-31.0.1-jre.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/ha-api-3.1.13.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/handlebars-4.3.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/httpclient-4.5.13.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/httpcore-4.4.15.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/ion-java-1.0.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/ipaddress-5.3.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/itext-2.1.7.js9.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/j2objc-annotations-1.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jackson-annotations-2.13.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jackson-core-2.13.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jackson-databind-2.13.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jackson-dataformat-cbor-2.13.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jai-imageio-core-1.4.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.activation-2.0.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.annotation-api-1.3.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.jws-api-2.1.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.mail-1.6.7.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.xml.bind-api-2.3.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.xml.soap-api-1.4.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jakarta.xml.ws-api-2.3.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jasperreports-6.18.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jasperreports-fonts-6.18.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/java-saml-2.8.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/java-saml-core-2.8.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/javax.activation-api-1.2.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/javax.annotation-api-1.3.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/javax.inject-1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/javax.mail-1.6.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/javax.xml.soap-api-1.4.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jaxb-api-2.3.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jaxb-core-2.3.0.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jaxb-impl-2.3.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jaxws-api-2.3.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jaxws-rt-2.3.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jbig2-imageio-3.0.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jcommon-1.0.24.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jempbox-1.8.16.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jettison-1.4.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jfreechart-1.0.19.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jmespath-java-1.12.145.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/joda-time-2.10.6.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jsch-0.1.55.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jsoup-1.14.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jsr181-api-1.0-MR1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/jsr305-3.0.2.jar
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>
> URL=file:/WEB-INF/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/log4j-api-2.17.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/log4j-core-2.17.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/log4j-slf4j-impl-2.17.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/log4j-web-2.17.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/management-api-3.2.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/metadata-extractor-2.16.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/mimepull-1.9.15.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/opencensus-api-0.28.0.jar
> >>>>>>>>>
> URL=file:/WEB-INF/lib/opencensus-contrib-http-util-0.28.0.jar
> >>>>>>>>>
> >>> URL=file:/WEB-INF/lib/owasp-java-html-sanitizer-20211018.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/pdfbox-2.0.25.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/perfmark-api-0.23.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/poi-4.1.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/poi-ooxml-4.1.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/poi-ooxml-schemas-4.1.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/policy-2.7.10.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/postgresql-42.3.1.jar
> >>>>>>>>>
> URL=file:/WEB-INF/lib/proto-google-cloud-speech-v1-1.30.5.jar
> >>>>>>>>>
> >>>>>> URL=file:/WEB-INF/lib/proto-google-cloud-speech-v1p1beta1-0.83.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/proto-google-common-protos-2.3.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/protobuf-java-3.17.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/protobuf-java-util-3.17.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/saaj-impl-1.5.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/scala-library-2.11.12.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/slf4j-api-1.7.33.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/stax-api-1.0.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/stax-ex-1.8.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/stax2-api-4.2.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/streambuffer-2.0.2.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/taglibs-standard-impl-1.2.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/taglibs-standard-spec-1.2.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/threetenbp-1.3.5.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/tika-core-2.2.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/tika-parser-image-module-2.2.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/tika-parser-xmp-commons-2.2.1.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/woodstox-core-6.2.6.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/xmlbeans-3.1.0.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/xmlsec-2.2.3.jar
> >>>>>>>>> URL=file:/WEB-INF/lib/xmpcore-6.1.11.jar
> >>>>>>>>> Class loader name=java.net.URLClassLoader, URL count=33
> >>>>>>>>> URL=file:/lib/
> >>>>>>>>> URL=file:/lib/annotations-api.jar
> >>>>>>>>> URL=file:/lib/catalina-ant.jar
> >>>>>>>>> URL=file:/lib/catalina-ha.jar
> >>>>>>>>> URL=file:/lib/catalina-ssi.jar
> >>>>>>>>> URL=file:/lib/catalina-storeconfig.jar
> >>>>>>>>> URL=file:/lib/catalina-tribes.jar
> >>>>>>>>> URL=file:/lib/catalina.jar
> >>>>>>>>> URL=file:/lib/ecj-4.20.jar
> >>>>>>>>> URL=file:/lib/el-api.jar
> >>>>>>>>> URL=file:/lib/jasper-el.jar
> >>>>>>>>> URL=file:/lib/jasper.jar
> >>>>>>>>> URL=file:/lib/jaspic-api.jar
> >>>>>>>>> URL=file:/lib/jsp-api.jar
> >>>>>>>>> URL=file:/lib/servlet-api.jar
> >>>>>>>>> URL=file:/lib/tomcat-api.jar
> >>>>>>>>> URL=file:/lib/tomcat-coyote.jar
> >>>>>>>>> URL=file:/lib/tomcat-dbcp.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-cs.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-de.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-es.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-fr.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-ja.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-ko.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-pt-BR.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-ru.jar
> >>>>>>>>> URL=file:/lib/tomcat-i18n-zh-CN.jar
> >>>>>>>>> URL=file:/lib/tomcat-jdbc.jar
> >>>>>>>>> URL=file:/lib/tomcat-jni.jar
> >>>>>>>>> URL=file:/lib/tomcat-util-scan.jar
> >>>>>>>>> URL=file:/lib/tomcat-util.jar
> >>>>>>>>> URL=file:/lib/tomcat-websocket.jar
> >>>>>>>>> URL=file:/lib/websocket-api.jar
> >>>>>>>>> Class loader
> >>> name=jdk.internal.loader.ClassLoaders$AppClassLoader
> >>>>>>>>> Class loader
> >>>>>> name=jdk.internal.loader.ClassLoaders$PlatformClassLoader
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> [3]
> >>>>>>>>> private static void
> >>>>>> logClassLoaderUrls(org.apache.logging.log4j.Level
> >>>>>>>>> logLevel, ClassLoader cl) {
> >>>>>>>>> if (null == cl) {
> >>>>>>>>> return;
> >>>>>>>>> }
> >>>>>>>>> boolean loaderNameLogged = false;
> >>>>>>>>> if (cl instanceof URLClassLoader) {
> >>>>>>>>> final URLClassLoader urlCl = (URLClassLoader) cl;
> >>>>>>>>> final URL[] urls = urlCl.getURLs();
> >>>>>>>>> if (null != urls) {
> >>>>>>>>> LOG.log(logLevel, " Class loader name={}, URL
> >>>>>> count={}",
> >>>>>>>>> cl.getClass().getName(), urls.length);
> >>>>>>>>> loaderNameLogged = true;
> >>>>>>>>> for (int idx = 0; idx < urls.length; idx++) {
> >>>>>>>>> final URL url = urls[idx];
> >>>>>>>>> LOG.log(logLevel, " URL[{}]={}", idx,
> >>> url);
> >>>>>>>>> }
> >>>>>>>>> }
> >>>>>>>>> }
> >>>>>>>>> if (!loaderNameLogged) {
> >>>>>>>>> LOG.log(logLevel, " Class loader name={}",
> >>>>>>>>> cl.getClass().getName());
> >>>>>>>>> }
> >>>>>>>>> logClassLoaderUrls(logLevel, cl.getParent());
> >>>>>>>>> }
> >>>>>>>>>
> >>>>>>>>> ..snip..
> >>>>>>>>>
> >>>>>>>>> LOG.info("java.class.path={}",
> >>>>>>>>> System.getProperty("java.class.path"));
> >>>>>>>>> LOG.debug("Class loader URLs:");
> >>>>>>>>>
> >>> logClassLoaderUrls(org.apache.logging.log4j.Level.DEBUG,
> >>>>>>>>> ctx.getClassLoader());
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> [4]
> >>>>>>>>> cat log-extract.txt | cut -c137- | sed -e
> >>>>>> "s+/usr/local/tomcat++g" -e
> >>>>>>>>> "s+/usr/local/tomcat/webapps/exclaim++g" -e "s+\[[0-9]*\]++"
> >>>>>>>>> and I manually sorted the "sections" using vim, with the "sort"
> >>>>>> command.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> [5]
> >>>>>>>>> # Debian with OpenJDK
> >>>>>>>>> #FROM tomcat:9.0.58-jre11-openjdk AS tomcat_package
> >>>>>>>>> # AWS Linux 2 with Corretto
> >>>>>>>>> FROM tomcat:9.0.58-jdk11-corretto AS tomcat_package
> >>>>>>>>>
> >>>>>>>>> # Set the timezone to Canada/Eastern
> >>>>>>>>> # For Debian
> >>>>>>>>> #RUN ln -sf /usr/share/zoneinfo/Canada/Eastern /etc/localtime &&
> >>> echo
> >>>>>>>>> "Canada/Eastern" > /etc/timezone && dpkg-reconfigure -f
> >>> noninteractive
> >>>>>>>>> tzdata
> >>>>>>>>> # For AWS Linux
> >>>>>>>>> RUN ln -sf /usr/share/zoneinfo/Canada/Eastern /etc/localtime &&
> >>> sed -i
> >>>>>>>>> 's/\"UTC\"/\"Canada\/Eastern\"/' /etc/sysconfig/clock # and
> reboot,
> >>>>>> but
> >>>>>>>>> will skip as container will be created and ran separately
> >>>>>>>>>
> >>>>>>>>> # Install a postgresql client for testing and the required font
> >>>>>> packages
> >>>>>>>>> needed by JasperReports
> >>>>>>>>> # For Debian:
> >>>>>>>>> #RUN echo "deb http://deb.debian.org/debian bullseye contrib
> >>>>>> non-free"
> >>>>>>>>>>> /etc/apt/sources.list && apt-get update && apt-get install -y
> >>>>>>>>> postgresql-client ttf-mscorefonts-installer
> >>>>>>>>> # For AWS Linux:
> >>>>>>>>> RUN yum -q list installed epel-release.noarch &>/dev/null && echo
> >>>>>>>>> 'epel-release already installed' || yum install -y
> >>>>>>>>>
> >>>>>>
> >>> https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
> >>>>>>>>> RUN yum install -y cabextract xorg-x11-font-utils fontconfig
> >>>>>>>>> RUN rpm -q msttcore-fonts-installer-2.6-1 &>/dev/null && echo
> >>>>>>>>> 'msttcore-fonts-installer-2.6-1 already installed' || rpm -i
> >>>>>>>>>
> >>>>>>
> >>>
> https://iweb.dl.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm
> >>>>>>>>>
> >>>>>>>>> # Add a health-check script to wait for exclaim to be running --
> >>>>>> allow at
> >>>>>>>>> least 10 minutes for it to start
> >>>>>>>>> COPY health_check.sh /usr/local/bin/
> >>>>>>>>> HEALTHCHECK --interval=10s --timeout=5s --start-period=10m
> >>>>>> --retries=10
> >>>>>>>>> CMD /usr/local/bin/health_check.sh
> >>>>>>>>>
> >>>>>>>>> # Set up the resource folders needed by application and make it a
> >>>>>>>>> # volume so it persists across container restarts
> >>>>>>>>> COPY resources/ /resources/
> >>>>>>>>> VOLUME /resources
> >>>>>>>>>
> >>>>>>>>> # Map some of the Tomcat folders to volumes so they persist
> between
> >>>>>>>>> restarts of the container
> >>>>>>>>> VOLUME /usr/local/tomcat/logs
> >>>>>>>>> VOLUME /usr/local/tomcat/webapps
> >>>>>>>>> VOLUME /usr/local/tomcat/work
> >>>>>>>>> VOLUME /usr/local/tomcat/conf
> >>>>>>>>>
> >>>>>>>>> # Enable connecting from anywhere for JPDA (req'd for Java 9+)
> >>>>>>>>> ENV JPDA_ADDRESS=*:8000
> >>>>>>>>>
> >>>>>>>>> CMD ["catalina.sh","jpda","run"]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> [6]
> >>>>>>>>> JDK on MacOS is:
> >>>>>>>>> openjdk version "11.0.13" 2021-10-19 LTS
> >>>>>>>>> OpenJDK Runtime Environment Corretto-11.0.13.8.1 (build
> >>> 11.0.13+8-LTS)
> >>>>>>>>> OpenJDK 64-Bit Server VM Corretto-11.0.13.8.1 (build
> 11.0.13+8-LTS,
> >>>>>> mixed
> >>>>>>>>> mode)
> >>>>>>>>>
> >>>>>>>>> JDK on Windows is:
> >>>>>>>>> openjdk 11.0.14 2022-01-18 LTS
> >>>>>>>>> OpenJDK Runtime Environment Corretto-11.0.14.9.1 (build
> >>> 11.0.14+9-LTS)
> >>>>>>>>> OpenJDK 64-Bit Server VM Corretto-11.0.14.9.1 (build
> 11.0.14+9-LTS,
> >>>>>> mixed
> >>>>>>>>> mode)
> >>>>>>>>>
> >>>>>>>>> JDK on Docker with AWS Linux 2 is:
> >>>>>>>>> openjdk 11.0.14 2022-01-18 LTS
> >>>>>>>>> OpenJDK Runtime Environment Corretto-11.0.14.9.1 (build
> >>> 11.0.14+9-LTS)
> >>>>>>>>> OpenJDK 64-Bit Server VM Corretto-11.0.14.9.1 (build
> 11.0.14+9-LTS,
> >>>>>> mixed
> >>>>>>>>> mode)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Feb 7, 2022 at 9:24 PM Rob Sargent <
> rsarg...@xmission.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 2/7/22 19:13, Robert Turner wrote:
> >>>>>>>>>>> So, I've gone back and double-checked as much as I can (Tomcat
> >>>>>>>>>> version, JDK
> >>>>>>>>>>> version), and the classpath, and I have identical classpaths in
> >>> both
> >>>>>>>>>>> environments (except the sort order of the URLs per "level" of
> >>>>>>>>>>> ClassLoader), and I've re-verified the behaviour:
> >>>>>>>>>>> - fails in the docker environment
> >>>>>>>>>>> - works locally
> >>>>>>>>>>> (Same WAR file on both).
> >>>>>>>>>>>
> >>>>>>>>>>> I guess I'm in to one of the following approaches next:
> >>>>>>>>>>> - build a debug version of Tomcat, and step through the
> code
> >>> that
> >>>>>>>>>> breaks
> >>>>>>>>>>> and see if I can figure out why (not that I have time to do
> this
> >>> of
> >>>>>>>>>>> course...but might be necessary)
> >>>>>>>>>>> - construct a trivial application reproduction, along with
> >>> docker
> >>>>>>>>>> layout,
> >>>>>>>>>>> and see if anyone else can reproduce... (assuming anyone else
> has
> >>>>>> time
> >>>>>>>>>> to
> >>>>>>>>>>> do that of course...)
> >>>>>>>>>>>
> >>>>>>>>>>> Anyone got any suggestions of what to look into next?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Feb 7, 2022 at 5:05 PM Rob Sargent <
> >>> rsarg...@xmission.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> As you appear averse to sharing names of things, seems you will
> >>> need
> >>>>>> to
> >>>>>>>>>> do a lot of extra clean-up before you can share a docker image.
> >>> Make
> >>>>>>>>>> sure you're clear on what NoClassDefFoundError indicates
> (present
> >>> at
> >>>>>>>>>> compile time, absent at runtime) and how that translates to
> >>>>>> docker-ness
> >>>>>>>>>> (of which I know naught).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> >>>>>>>>>> For additional commands, e-mail: users-h...@tomcat.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> >>>>>> For additional commands, e-mail: users-h...@tomcat.apache.org
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> >>> For additional commands, e-mail: users-h...@tomcat.apache.org
> >>>
> >>>
> >
>
