Tomcat SSL Connector - Http11NioProtocol - javax.crypto.ShortBufferException on second request

2020-04-13 Thread Parigino Andrea Aiello
Hello!
i'm having a problem with Tomcat 8.5.51 hosting my Spring Boot 2
application (with 2-way SSL);
In short is an application with both server and client SOAP interfaces
(first called as server, then it act as client).
The problem:
on first request (sent by SoapUI or other external client) everything works
fine, no exception;
on the second one i got this exception:

   1. 13-Apr-2020 11:45:09.757 INFO [https-jsse-nio-234-exec-1]
   org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request
   header
Note: further occurrences of HTTP request parsing errors will be logged
   at DEBUG level.
   java.lang.ArrayIndexOutOfBoundsException:
   javax.crypto.ShortBufferException: Need at least 336 bytes of space in
   output buffer
   at sun.security.ssl.CipherBox.decrypt(CipherBox.java:591)
   at
   sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:200)
   at
   sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:963)
   at
   sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
   at
   sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
   at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
   at
   org.apache.tomcat.util.net.SecureNioChannel.read(SecureNioChannel.java:607)
   at
   
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1289)
   at
   
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1225)
   at
   org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:737)
   at
   
org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:368)
   at
   org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:502)
   at
   
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
   at
   
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
   at
   
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1623)
   at
   
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at
   
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at
   
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at
   
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.lang.Thread.run(Thread.java:748)

To be noted that on the second request i do not get even a single line of
log from my application, looks like the request doesn't even reach my code.
here is the Connector config:



i've also tried all the buffer parameter for the connector (
https://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html#NIO_specific_configuration
--> setting them to -1/illimited) but seem to not work.

Another thing to say is that between the acting as SOAP Server and acting
SOAP Client there are some http (not https) calls to another system.

Any help would be really appreciated.
Thanks a lot!

Andrea


Re: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986

2020-04-13 Thread Manuel Dominguez Sarmiento
Sorry, I was not aware of that behaviour even when changing the subject. 
I'll send a new, separate, unrelated message.

*
Manuel Dominguez Sarmiento*

On 12/04/2020 16:08, Mark Thomas wrote:

Please don't hijack an existing thread. Start a new message for a new
topic. (Replying to an existing message and changing the header is not
sufficient.)

Mark


On 09/04/2020 21:05, Manuel Dominguez Sarmiento wrote:

Hi, we're reviewing our logs, are we are ocasionally getting the
following stack traces:

09-Apr-2020 11:29:19.489 INFO [tomcat-http-81]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
request header
  Note: further occurrences of HTTP request parsing errors will be logged
at DEBUG level.
     java.lang.IllegalArgumentException: Invalid character found in
the request target. The valid characters are defined in RFC 7230 and RFC
3986
     at
org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:488)

     at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:260)
     at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)

     at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)

     at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1594)

     at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

     at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)

     at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)

     at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)


We understand this is a consequence of malformed requests, but we cannot
seem to pinpoint the cause. It seems these are clients outside of our
control (our servers are public-facing). The AccessLogValve does not log
these requests, so we cannot figure out what the request line is. Is
there any way logging could be improved in order to find out what is
causing this?

BTW, we're on Tomcat 9.0.33

*Manuel Dominguez Sarmiento*




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986

2020-04-13 Thread Manuel Dominguez Sarmiento
Hi, we're reviewing our logs, are we are ocasionally getting the 
following stack traces:


09-Apr-2020 11:29:19.489 INFO [tomcat-http-81] 
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP 
request header
 Note: further occurrences of HTTP request parsing errors will be 
logged at DEBUG level.
    java.lang.IllegalArgumentException: Invalid character found in 
the request target. The valid characters are defined in RFC 7230 and RFC 
3986
    at 
org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:488)
    at 
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:260)
    at 
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at 
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
    at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1594)
    at 
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)


We understand this is a consequence of malformed requests, but we cannot 
seem to pinpoint the cause. It seems these are clients outside of our 
control (our servers are public-facing). The AccessLogValve does not log 
these requests, so we cannot figure out what the request line is. Is 
there any way logging could be improved in order to find out what is 
causing this?


BTW, we're on Tomcat 9.0.33

*Manuel Dominguez Sarmiento*


Re: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986

2020-04-13 Thread Mark Thomas
On 13/04/2020 17:25, Manuel Dominguez Sarmiento wrote:
> Hi, we're reviewing our logs, are we are ocasionally getting the
> following stack traces:
> 
> 09-Apr-2020 11:29:19.489 INFO [tomcat-http-81]
> org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
> request header
>  Note: further occurrences of HTTP request parsing errors will be logged
> at DEBUG level.
>     java.lang.IllegalArgumentException: Invalid character found in
> the request target. The valid characters are defined in RFC 7230 and RFC
> 3986
>     at
> org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:488)



> We understand this is a consequence of malformed requests, but we cannot
> seem to pinpoint the cause. It seems these are clients outside of our
> control (our servers are public-facing). The AccessLogValve does not log
> these requests, so we cannot figure out what the request line is. Is
> there any way logging could be improved in order to find out what is
> causing this?

The stack trace indicates the problem is in the query string if that helps.

Yes, I think we should be able to do something here. The tricky part is
that as soon as an invalid character is detected we have to be a lot
more careful as the payload could be malicious. I'm not sure if we'll be
able to get anything into the access log but it should be possible to
improve the error message and include the problematic request line in
some form. You probably won't see the exact request line as we'll need
to encoded things like control characters etc.

I'll look at this for the May round of releases.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat SSL Connector - Http11NioProtocol - javax.crypto.ShortBufferException on second request

2020-04-13 Thread Mark Thomas
On 13/04/2020 11:39, Parigino Andrea Aiello wrote:
> Hello!
> i'm having a problem with Tomcat 8.5.51 hosting my Spring Boot 2
> application (with 2-way SSL);

The first thing to do is to update to 8.5.54 and re-test.

Mark

> In short is an application with both server and client SOAP interfaces
> (first called as server, then it act as client).
> The problem:
> on first request (sent by SoapUI or other external client) everything works
> fine, no exception;
> on the second one i got this exception:
> 
>1. 13-Apr-2020 11:45:09.757 INFO [https-jsse-nio-234-exec-1]
>org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request
>header
> Note: further occurrences of HTTP request parsing errors will be logged
>at DEBUG level.
>java.lang.ArrayIndexOutOfBoundsException:
>javax.crypto.ShortBufferException: Need at least 336 bytes of space in
>output buffer
>at sun.security.ssl.CipherBox.decrypt(CipherBox.java:591)
>at
>sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:200)
>at
>sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:963)
>at
>sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
>at
>sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
>at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
>at
>org.apache.tomcat.util.net.SecureNioChannel.read(SecureNioChannel.java:607)
>at
>
> org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1289)
>at
>
> org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1225)
>at
>org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:737)
>at
>
> org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:368)
>at
>org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:502)
>at
>
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
>at
>
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
>at
>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1623)
>at
>
> org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
>at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>at
>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>at java.lang.Thread.run(Thread.java:748)
> 
> To be noted that on the second request i do not get even a single line of
> log from my application, looks like the request doesn't even reach my code.
> here is the Connector config:
> 
>  sslImplementationName="org.apache.tomcat.util.net.jsse.JSSEImplementation"
> port="234" maxThreads="200" scheme="https" secure="true"
> SSLEnabled="true" clientAuth="true" sslProtocol="TLS"
> keyAlias="agweb2ca"
> keystoreFile="conf\cert\keystore_s.jks" keystorePass="*"
> truststoreFile="conf\cert\truststore_s.jks" truststorePass="**"
> />
> 
> i've also tried all the buffer parameter for the connector (
> https://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html#NIO_specific_configuration
> --> setting them to -1/illimited) but seem to not work.
> 
> Another thing to say is that between the acting as SOAP Server and acting
> SOAP Client there are some http (not https) calls to another system.
> 
> Any help would be really appreciated.
> Thanks a lot!
> 
> Andrea
> 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: Alternatives for AJP

2020-04-13 Thread David Cleary
https://nvd.nist.gov/vuln/detail/CVE-2020-1938

-Original Message-
From: stephane passignat  
Sent: Sunday, April 12, 2020 4:00 AM
To: Tomcat Users List 
Subject: Re: Alternatives for AJP

Hi
Which vulnerability are you mentioning ?
Thanks

⁣Envoyé par BlueMail ​

Le 10 avr. 2020 à 17:45, à 17:45, David Cleary  a écrit:
>Some of our customers are currently using the AJP connector. Given the 
>vulnerability and breaking change to address it, now may be a good time 
>to prompt them look at alternatives. One requirement is HTTPS support.
>What are the alternatives when hosting Tomcat behind Apache httpd, 
>nginx, or IIS? I do remember a presentation I thought was pretty good 
>at Apachecon in Miami on connectors a few years ago. Has there been 
>anything new that has come out since then? Are there any 
>recommendations on what is best to replace AJP13?
>
>Thanks
>Dave


Re: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986

2020-04-13 Thread Manuel Dominguez Sarmiento
Thanks Mark. Including the request line (encoded if necessary to avoid 
issues with control characters) should definitely help.
Getting through all the way to AccessLogValve would also help, but the 
most important bit is improving the error message.


*Manuel Dominguez Sarmiento*

On 13/04/2020 14:04, Mark Thomas wrote:

On 13/04/2020 17:25, Manuel Dominguez Sarmiento wrote:

Hi, we're reviewing our logs, are we are ocasionally getting the
following stack traces:

09-Apr-2020 11:29:19.489 INFO [tomcat-http-81]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
request header
  Note: further occurrences of HTTP request parsing errors will be logged
at DEBUG level.
     java.lang.IllegalArgumentException: Invalid character found in
the request target. The valid characters are defined in RFC 7230 and RFC
3986
     at
org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:488)




We understand this is a consequence of malformed requests, but we cannot
seem to pinpoint the cause. It seems these are clients outside of our
control (our servers are public-facing). The AccessLogValve does not log
these requests, so we cannot figure out what the request line is. Is
there any way logging could be improved in order to find out what is
causing this?

The stack trace indicates the problem is in the query string if that helps.

Yes, I think we should be able to do something here. The tricky part is
that as soon as an invalid character is detected we have to be a lot
more careful as the payload could be malicious. I'm not sure if we'll be
able to get anything into the access log but it should be possible to
improve the error message and include the problematic request line in
some form. You probably won't see the exact request line as we'll need
to encoded things like control characters etc.

I'll look at this for the May round of releases.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





Re: Tomcat SSL Connector - Http11NioProtocol - javax.crypto.ShortBufferException on second request

2020-04-13 Thread Rémy Maucherat
On Mon, Apr 13, 2020 at 7:07 PM Mark Thomas  wrote:

> On 13/04/2020 11:39, Parigino Andrea Aiello wrote:
> > Hello!
> > i'm having a problem with Tomcat 8.5.51 hosting my Spring Boot 2
> > application (with 2-way SSL);
>
> The first thing to do is to update to 8.5.54 and re-test.
>

Also test OpenSSL and Java 11 [if Java 8 was used here], to see what
happens.

Rémy


>
> Mark
>
> > In short is an application with both server and client SOAP interfaces
> > (first called as server, then it act as client).
> > The problem:
> > on first request (sent by SoapUI or other external client) everything
> works
> > fine, no exception;
> > on the second one i got this exception:
> >
> >1. 13-Apr-2020 11:45:09.757 INFO [https-jsse-nio-234-exec-1]
> >org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
> request
> >header
> > Note: further occurrences of HTTP request parsing errors will be
> logged
> >at DEBUG level.
> >java.lang.ArrayIndexOutOfBoundsException:
> >javax.crypto.ShortBufferException: Need at least 336 bytes of space in
> >output buffer
> >at
> sun.security.ssl.CipherBox.decrypt(CipherBox.java:591)
> >at
> >sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:200)
> >at
> >sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:963)
> >at
> >sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
> >at
> >sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
> >at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
> >at
> >org.apache.tomcat.util.net
> .SecureNioChannel.read(SecureNioChannel.java:607)
> >at
> >org.apache.tomcat.util.net
> .NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1289)
> >at
> >org.apache.tomcat.util.net
> .NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1225)
> >at
> >
> org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:737)
> >at
> >
> org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:368)
> >at
> >
> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:502)
> >at
> >
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> >at
> >
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
> >at
> >org.apache.tomcat.util.net
> .NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1623)
> >at
> >org.apache.tomcat.util.net
> .SocketProcessorBase.run(SocketProcessorBase.java:49)
> >at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >at
> >
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> >at java.lang.Thread.run(Thread.java:748)
> >
> > To be noted that on the second request i do not get even a single line of
> > log from my application, looks like the request doesn't even reach my
> code.
> > here is the Connector config:
> >
> >  >
> sslImplementationName="org.apache.tomcat.util.net.jsse.JSSEImplementation"
> > port="234" maxThreads="200" scheme="https" secure="true"
> > SSLEnabled="true" clientAuth="true" sslProtocol="TLS"
> > keyAlias="agweb2ca"
> > keystoreFile="conf\cert\keystore_s.jks" keystorePass="*"
> > truststoreFile="conf\cert\truststore_s.jks" truststorePass="**"
> > />
> >
> > i've also tried all the buffer parameter for the connector (
> >
> https://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html#NIO_specific_configuration
> > --> setting them to -1/illimited) but seem to not work.
> >
> > Another thing to say is that between the acting as SOAP Server and acting
> > SOAP Client there are some http (not https) calls to another system.
> >
> > Any help would be really appreciated.
> > Thanks a lot!
> >
> > Andrea
> >
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>


AccessLogValve and IPv6 string representation (RFC 5952 section 4)

2020-04-13 Thread Manuel Dominguez Sarmiento
Hi, we are in the middle of a thorough review to fully support IPv6 
across our platform. It has come to our attention that Java does not 
fully conform to RFC 5952 section 4 which deals with IPv6 zero 
compression (i.e. ::1 instead of 0:0:0:0:0:0:0:1 for localhost). We have 
confirmed that Tomcat's AccessLogValve is using the standard Java 
implementation. How can we guarantee zero compression to be used in 
AccessLogValve?


We are using Guava's InetAddresses.toAddrString() across our systems to 
deal with this. We know we can use a custom AccessLogValve extending the 
standard behaviour, but we were wondering whether there was any other 
solution, option or flag around this. We've thought of using a custom 
request attribute to hold the IP address, but this is not very elegant. 
In particular, we'd lose the IP address if the filter we would use to 
set the request attribute is not invoked for any reason.


This is not minor, since we use access logs a lot to diagnose issues, 
and cross-reference IP addresses with many other systems which are fully 
RFC 5952-compliant. Having separate representations for the same IP 
address will eventually lead to either trouble, misdiagnosis, missed 
records, etc.


Any suggestions?

*Manuel Dominguez Sarmiento*



Re: AccessLogValve and IPv6 string representation (RFC 5952 section 4)

2020-04-13 Thread Michael Osipov

Am 2020-04-14 um 01:45 schrieb Manuel Dominguez Sarmiento:
Hi, we are in the middle of a thorough review to fully support IPv6 
across our platform. It has come to our attention that Java does not 
fully conform to RFC 5952 section 4 which deals with IPv6 zero 
compression (i.e. ::1 instead of 0:0:0:0:0:0:0:1 for localhost). We have 
confirmed that Tomcat's AccessLogValve is using the standard Java 
implementation. How can we guarantee zero compression to be used in 
AccessLogValve?


Are you explicitly referring to:

4.2.1.  Shorten as Much as Possible

   The use of the symbol "::" MUST be used to its maximum capability.
   For example, 2001:db8:0:0:0:0:2:1 must be shortened to 2001:db8::2:1.
   Likewise, 2001:db8::0:1 is not acceptable, because the symbol "::"
   could have been used to produce a shorter representation 2001:db8::1.


Have you considered raising with net-...@openjdk.java.net?


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: AccessLogValve and IPv6 string representation (RFC 5952 section 4)

2020-04-13 Thread Manuel Dominguez Sarmiento
Nevermind. For some reason we had omitted this is already supported by the 
ipv6Canonical flag. 

RTFM!

Manuel Dominguez Sarmiento


> On 13 Apr 2020, at 20:46, Manuel Dominguez Sarmiento  wrote:
> 
>  Hi, we are in the middle of a thorough review to fully support IPv6 across 
> our platform. It has come to our attention that Java does not fully conform 
> to RFC 5952 section 4 which deals with IPv6 zero compression (i.e. ::1 
> instead of 0:0:0:0:0:0:0:1 for localhost). We have confirmed that Tomcat's 
> AccessLogValve is using the standard Java implementation. How can we 
> guarantee zero compression to be used in AccessLogValve?
> 
> We are using Guava's InetAddresses.toAddrString() across our systems to deal 
> with this. We know we can use a custom AccessLogValve extending the standard 
> behaviour, but we were wondering whether there was any other solution, option 
> or flag around this. We've thought of using a custom request attribute to 
> hold the IP address, but this is not very elegant. In particular, we'd lose 
> the IP address if the filter we would use to set the request attribute is not 
> invoked for any reason.
> 
> This is not minor, since we use access logs a lot to diagnose issues, and 
> cross-reference IP addresses with many other systems which are fully RFC 
> 5952-compliant. Having separate representations for the same IP address will 
> eventually lead to either trouble, misdiagnosis, missed records, etc.
> 
> Any suggestions?
> 
> Manuel Dominguez Sarmiento
> 


JNDI match of LDAP hashed passwords fail against cleartext

2020-04-13 Thread Brian Burch
I thought it would be helpful to start this issue on the users list 
because it will contain a lot of helpful search terms.


I am upgrading a stable production tomcat 7.0.52 system to tomcat 
8.5.54. Both were built from source code (tc8 cloned from git) and 
compiled under openjdk8.


Many users have pre-hashed SHA-1 passwords stored in the LDAP directory. 
My SSO login jsp uses Form authentication. Because the only Connector 
services https on port 443, there is no security exposure from sending 
the user-entered cleartext password "over the wire" to tomcat.


The working tomcat 7 Engine has the following Realm definition:-

  

  
   connectionName="uid=tomcatAuthenticate,ou=Special 
Users,o=pingtoo.com"

 connectionPassword=""
 connectionURL="ldap://ldap.pingtoo.com:10389";
 userBase="ou=people,o=pingtoo.com"
 userSubtree="false"
 userSearch="(uid={0})"
 userRoleName="tomcatRole"
 userPassword="userPassword"
 digest="SHA" />
  

... and the Host has the following:-

className="org.apache.catalina.valves.ExtendedAccessLogValve"

   directory="logs"
   prefix="access." suffix=".txt"
   pattern="c-ip x-H(authType) x-H(remoteUser) date time 
cs-method cs-uri sc-status bytes"

   resolveHosts="false"/>




As I said earlier, but without sounding like I am complaining, this 
environment worked as intended (by me, the webadmin) with tomcat7.0.52.


I am aware of the following information on the tomcat 8 Documentation:-

https://tomcat.apache.org/tomcat-8.0-doc/realm-howto.html

and

https://tomcat.apache.org/tomcat-8.0-doc/config/realm.html#JNDI_Directory_Realm_-_org.apache.catalina.realm.JNDIRealm

It states the "algorithm attribute is deprecated. Set the algorithm on a 
nested CredentialHandler element instead."


I deleted the algorithm attribute from the tomcat8 JNDIRealm definition, 
and nested three different versions for different test runs:-


className="org.apache.catalina.realm.MessageDigestCredentialHandler"

   algorithm="SHA" />

It made no difference whether I used "SHA-1" or "SHA-256".

Under a netbeans remote debugging session with the tomcat8 server, I 
initially set a breakpoint in FormAuthenticator.doAuthenticate, then 
instruction stepped to drill down into the JNDI authentication logic.


JNDIREalm.compareCredentials executes:-

return getCredentialHandler().matches(credentials, password);

.. where the calling parameters are the {SHA} password from the 
LDAP directory and the cleartext password string from the logon Form.


MessageDigestCredentialHandler.matches immediately calls its own 
getAlgorithm method, which returns null. Without a correct digest 
handler, the two password parameters are compared as simple strings. 
This means the authentication fails!


To prove my point, I pasted the hashed copy of the LDAP userpassword (as 
plain text) into the FormAuthenticator field. Naturally, without a valid 
hash algorithm the authentication is successful simply because the two 
strings match exactly.


I searched for usages of MessageDigestCredentialHandler.setAlgorithm, 
but only found it used once - within TestJNDIRealm. I did not find any 
occurrences within tomcat mainline code, but would not be surprised if 
the algorithm was intended to be set within code which used 
introspection at runtime.


My initial code inspection makes me strongly suspect tomcat does not 
initialise JNDIRealm and a nested CredentialHandler properly during 
startup. However, I am not smart enough to attach my debugger to the 
tomcat jvm until it is too late.


I had a smart idea... at a breakpoint I changed the value of the 
algorithm instance variable from null to "SHA" before the comparison, 
but I was slapped down with the following Exception:-


BB 2020-04-14T15:22:44,257 ERROR 
[org.apache.catalina.core.ContainerBase.[Catalina].[www2.pingtoo.com]] 
Exception Processing /staticPingToo/restricted/j_security_check

java.lang.IllegalStateException: Must call init() first
at 
org.apache.tomcat.util.security.ConcurrentMessageDigest.digest(ConcurrentMessageDigest.java:71) 
~[tomcat-util.jar:8.5.54]
at 
org.apache.tomcat.util.security.ConcurrentMessageDigest.digest(ConcurrentMessageDigest.java:63) 
~[tomcat-util.jar:8.5.54]
at 
org.apache.catalina.realm.MessageDigestCredentialHandler.matches(MessageDigestCredentialHandler.java:114) 
~[catalina.jar:8.5.54]
at 
org.apache.catalina.realm.JNDIRealm.compareCredentials(JNDIRealm.java:1822) 
~[catalina.jar:8.5.54]
at 
org.apache.catalina.realm.JNDIRealm.checkCredentials(JNDIRealm.java:1782) 
~[catalina.jar:8.5.54]
at 
org.apache.catalina.realm.JNDIRealm.authenticate(JNDIRealm.java:1427) 
~[catalina.jar:8.5.54]
at 
org.apache.catalina.realm.JNDIRealm.auth