-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
All,
I believe I have determined the cause (or at least the fix) for this:
Despite the mod_proxy_ajp documentation, it is using packets larger
than 8192 bytes.
Setting this on the Tomcat <Connector> resolves the problem:
packetSize="65536"
I don't see anywhere you can set the maximum packet size on
mod_proxy_ajp so I think this is the only recourse.
Is anyone familiar enough with the mod_proxy_ajp code to know how to
look for this?
It was trivial to reproduce: just send a large POST message through
mod_proxy_ajp to Tomcat.
Thanks,
- -chris
On 6/25/20 10:28, Christopher Schultz wrote:
> All,
>
> This issue is apparently trivially reproducible in my dev
> environment.
>
> Do I have to get a protocol-trace to get any more helpful
> information?
>
> Thanks, -chris
>
> On 6/24/20 10:46, Christopher Schultz wrote:
>> All,
>
>> On 6/24/20 10:29, Christopher Schultz wrote:
>>> All,
>
>>> I'm slowly switching from mod_jk to mod_proxy_ajp and I have a
>>> development environment where I'm getting Bad Gateway
>>> responses sent to clients along with this exception in my
>>> Tomcat log file:
>
>>> java.lang.IllegalArgumentException: Header message of length
>>> [8,194] received but the packetSize is only [8,192] at
>>> org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:685
)
>
>>>
>>>
>
>> at
>>> org.apache.coyote.ajp.AjpProcessor.receive(AjpProcessor.java:626)
>>>
>>>
>
>>>
at
>>> org.apache.coyote.ajp.AjpProcessor.refillReadBuffer(AjpProcessor.jav
a
>
>>>
:
>
>>>
> 73
>
>
>> 4)
>>> at
>>> org.apache.coyote.ajp.AjpProcessor$SocketInputBuffer.doRead(AjpProce
s
>
>>>
s
>
>>>
> or
>
>
>> .java:1456)
>>> at org.apache.coyote.Request.doRead(Request.java:581) at
>>> org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.
j
>
>>>
a
>
>>>
> va
>
>
>> :344)
>>> at
>>> org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBu
f
>
>>>
f
>
>>>
> er
>
>
>> .java:663)
>>> at
>>> org.apache.catalina.connector.InputBuffer.readByte(InputBuffer.java:
3
>
>>>
5
>
>>>
> 8)
>
>
>> at
>>> org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStre
a
>
>>>
m
>
>>>
> .j
>
>
>> ava:93)
>>> at
>>> org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.j
a
>
>>>
v
>
>>>
> a:
>
>
>> 53)
>>> at
>>> org.apache.commons.io.input.TeeInputStream.read(TeeInputStream.java:
1
>
>>>
0
>
>>>
> 6)
>
>
>> at java.io.FilterInputStream.read(FilterInputStream.java:83)
>>> at my.product.MacInputStream.read(MacInputStream.java:29) at
>>> java.io.FilterInputStream.read(FilterInputStream.java:83) at
>>> com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableI
n
>
>>>
p
>
>>>
> ut
>
>
>> Stream.read(XMLEntityManager.java:2890)
>>> at
>>> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurren
t
>
>>>
E
>
>>>
> nt
>
>
>> ity(XMLEntityManager.java:674)
>>> at
>>> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determine
D
>
>>>
o
>
>>>
> cV
>
>
>> ersion(XMLVersionDetector.java:148)
>>> at
>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(
X
>
>>>
M
>
>>>
> L1
>
>
>> 1Configuration.java:806)
>>> at
>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(
X
>
>>>
M
>
>>>
> L1
>
>
>> 1Configuration.java:771)
>>> at
>>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser
.
>
>>>
j
>
>>>
> av
>
>
>> a:141)
>>> at
>>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(A
b
>
>>>
s
>
>>>
> tr
>
>
>> actSAXParser.java:1213)
>>> at
>>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.
p
>
>>>
a
>
>>>
> rs
>
>
>> e(SAXParserImpl.java:643)
>>> at
>>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParse
r
>
>>>
I
>
>>>
> mp
>
>
>> l.java:327)
>>> at javax.xml.parsers.SAXParser.parse(SAXParser.java:195)
>
>>> This is a web service which is reading the request with a
>>> SAXParser. It's been running in production (and dev!) for
>>> years without any issues. It''s been running for a few months
>>> in development, now, with mod_proxy_ajp without any errors.
>
>>> I know about the "max packet size" and the default is 8192
>>> bytes. I haven't changed the default. Here's my <Connector>
>>> configuration:
>
>>> <Connector port="8245" address="127.0.0.1"
>>> secretRequired="false" redirectPort="443" protocol="AJP/1.3"
>>> URIEncoding="UTF-8" executor="tomcatThreadPool" />
>
>>> Here's the configuration in httpd.conf:
>
>>> <Proxy "balancer://my-api"> BalancerMember
>>> "ajp://localhost:8245" timeout=300 ping=5 ttl=60 </Proxy>
>
>>> ProxyPass "/my-api/" "balancer://my-api/my-api/"
>>> ProxyPassReverse "/my-api/" "balancer://my-api/my-api/"
>
>>> The documentation for mod_proxy_ajp[1] seems to indicate that
>>> the "Packet Size" for AJP is fixed at 8192 bytes:
>
>>> " Packet Size
>
>>> According to much of the code, the max packet size is 8 * 1024
>>> bytes (8K). The actual length of the packet is encoded in the
>>> header.
>
>>> Packet Headers
>
>>> Packets sent from the server to the container begin with
>>> 0x1234. Packets sent from the container to the server begin
>>> with AB (that's the ASCII code for A followed by the ASCII code
>>> for B). After those first two bytes, there is an integer
>>> (encoded as above) with the length of the payload. Although
>>> this might suggest that the maximum payload could be as large
>>> as 2^16, in fact, *the code sets the maximum to be 8K*. "
>>> (emphasis mine)
>
>>> Does anyone know under what circumstances mod_proxy_ajp might
>>> send more than 8192 bytes? It looks like mod_proxy_ajp doesn't
>>> have any way to set the max packet size like mod_jk does.
>
>>> I should probably be able to set the max packet size on the
>>> Tomcat side to something higher than 8192 to catch this kind
>>> of thing... but it looks like it might be a bug in
>>> mod_proxy_ajp.
>
>>> Versions are Apache httpd 2.4.25 (Debian) and Tomcat 8.5.trunk
>>> (8.5.55). mod_jk is not being used.
>
>>> Any ideas?
>
>>> -chris
>
>>> [1] https://httpd.apache.org/docs/2.4/mod/mod_proxy_ajp.html
>
>
>> Some additional information:
>
>> 1. The headers of the HTTP request seem to be arriving in a
>> correct packet before this error occurs. The headers are only a
>> few hundred bytes (~340) and the request line should be
>> relatively short (~50 bytes or so). Method is POST, protocol is
>> HTTP/1.1.
>
>> 2. Apache httpd is terminating TLS. I have no configuration for
>> forwarding TLS information over to Tomcat, so I'm assuming it's
>> not being sent as part of the first packet.
>
>> 3. Before I get the packet-too-large error, I get another error:
>
>> org.apache.catalina.connector.ClientAbortException:
>> java.io.IOException: Invalid message received with length [-1]
>> at
>> org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.j
a
>
>>
va
>
>
> :348)
>> at
>> org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBuf
f
>
>>
er
>
>
> .java:663)
>> at
>> org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:370)
>
>>
>
> at
>> org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStrea
m
>
>>
.j
>
>
> ava:183)
>> at
>> org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.ja
v
>
>>
a:
>
>
> 75)
>> at
>> org.apache.commons.io.input.TeeInputStream.read(TeeInputStream.java:1
2
>
>>
4)
>
>
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
>> at my.product.MacInputStream.read(MacInputStream.java:49) at
>> java.io.FilterInputStream.read(FilterInputStream.java:107) at
>> my.product.XMLMessageProcessor.validate(XMLMessageProcessor.java:326)
>
>>
>
> at com.chadis.servlet.APIServlet.doPost(APIServlet.java:291)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
>
>> I haven't changed any configuration, yet. But if the first error
>> is size=-1 then it's unlikely that the problem will be solved by
>> increasing my max packet size on the Tomcat end.
>
>> I'm working with my client to see if we can reproduce this
>> problem reliably. I'm sure I can get more information once I do
>> that.
>
>> -chris
>
>
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl76FUoACgkQHPApP6U8
pFh+VRAAsxgojDBxkBhZe827nNXA5BC/Y7FZQCn6y2HxvQnJ7GsGVednrtQpNZPb
QgoTTKwu0OFkDyGLAaZ6pqRevUWbYB/Tin8wLe9+fh/pefg/E1W56zTTJFrGoa7A
FB/bKiF1t5C4EaC9k/i0i3MEZVuro6JxygmqZz54lttQS9OocQ2CM9/e3DIULkIQ
EX1lWAX0fwOJmYzO8RWuKktoKyagwk0SxTG40azEfCRYezXIsW58IDXPc2vS1+V6
FiiL0OEj8XKxZ2kMlpswZskrS7HvW9LJTCsFAsOC2KRYKA7Lwi8LaUtEYxDonBip
3XMKwzuokhi8Xt2r3fdQnLWiuAwnUDb5YC4OWXrS08lJ0OC1j7dnWd+C78QwtWDX
n+fAzhuDpeCMF0pfZsYd0+bjywMkebXhnkU05oP/WEySpuVJg1Q+1NrXd/2xQi6o
4QAnTuJc56M3fhr9UGy9i0mOo/jCOqFwutDGUPg/byktt7gUOXNDOUAS5gRTTQzz
mUvzKBxJJ/ipPjhtPVzsCzIE3xPHMJ9ttXn9Qd89anifwfBGrOyoMSOc4pGATOzv
hHCVhhvlPjOY04umZ4aqkRAnK8+fdmsRh0SHxt+VJNEnVs2I3h6oBDRuHEhPMHIb
D4Pjv5J0cNmGZ0rapSGy+v6qdPuICPw0SZPl0Ik0C7ft1/JZEFw=
=NLiq
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
