-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Kohei,
On 12/1/18 10:06, Kohei Nozaki wrote: > Hello Konstantin, thanks for sharing the valuable information. > >> On Dec 1, 2018, at 19:00, Konstantin Kolinko >> <knst.koli...@gmail.com> wrote: >> >>> * Our downstream Nginx instance (The client of our Tomcat >>> instance) recorded the error "upstream sent no valid HTTP/1.0 >>> header while reading response header from upstream" at that >>> time and the error makes perfect sense concerning the response >>> which has neither HTTP status line nor HTTP headers. >> >> 1. See the official FAQ / Troubleshoting page: >> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics >> >> >> Especially pay attention to >> 1) configuring an access log 2) setting system property >> org.apache.catalina.connector.RECYCLE_FACADES=true > > I've investigated "RECYCLE_FACADES" and understand that an > improperly implemented application which keeps a reference to > Request or Response objects outside of their lifecycle can make > such an issue like mine happen (please correct me if I'm wrong..). > > > But I still don't quite understand what does "RECYCLE_FACADES=true" > do. The Wiki page says "This makes it easier to spot illegal access > when it happens, instead of waiting until side effects of such > access become visible" but how does it make easier? Does this > property make Tomcat produce an Exception or make Tomcat produce > some warning message to a log file or something when such accesses > happen, for example? Tomcat usually handles requests something like this. Imagine a single-threaded server where Tomcat only accepts a single connection at a time (just to simplify the code to the point where it fits into a ML post). Many of these methods are made-up. There is no TomcatHttpServletRequest class or a .setRequestLine method in it (though there are ... siilar concepts in there, way down deep). The point is how the objects are used, or rather *re* used. HttpServletRequest request = new TomcatHttpServletRequest(); HttpServletResponse response = new TomcatHttpServletResponse(); Connection conn = null; while(null != (conn = socket.acceptConnection()) { request.setRequestLine(conn.getRequestLine()); request.setInputStream(conn.getInputStream()); response.setOutputStream(conn.getOutputStream()); Servlet servlet = getServletForRequest(request); if(null == servlet) servlet = defaultServlet; servlet.service(request, response); request.reset(); response.reset(); } In "real" Tomcat, each Connection object holds its own Request and Response objects ad manages them in a similar way, and of course, Tomcat can accept multiple simultaneous connections -- including multiple requests over a single connection -- simultaneously -- in the case of HTTP/2. If you enable the RECYCLE_FACADES in Tomcat, the code changes to behave like this: Connection conn = null; while(null != (conn = socket.acceptConnection()) { HttpServletRequest request = new TomcatHttpServletRequest(); HttpServletResponse response = new TomcatHttpServletResponse(); request.setRequestLine(conn.getRequestLine()); request.setInputStream(conn.getInputStream()); response.setOutputStream(conn.getOutputStream()); Servlet servlet = getServletForRequest(request); if(null == servlet) servlet = defaultServlet; servlet.service(request, response); request.dispose(); response.dispose(); } Note how the request and response objects are no longer re-used across requests. This represents a trade-off between security/stability (always getting a fresh object) versus performance (less garbage-collection for a given request). An application can do things to the request or response that can break the way the server works, so an untrusted application should always be run with RECYCLE_FACADES set to ON. If the application keeps a reference to a request or response object after the request has completed (as defined by returning from servlet.service()), then Bad Things can happen. If you write to that response's OutputStream, for example, you might write into the response of *another request* that is being handled after your code should be finished. > > p.s. Speaking of the possibility of an improper implementation, > I've found some discussions which seem to have some similarity to > my case like the following: > > * > http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2 054107.html > : I've seen some similar Exceptions at the "MimeHeaders" class > which were recorded in my Tomcat's log too > > * > http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-respons es-td5034710.html > : A case where a bug in a Filter make Tomcat produce an invalid > HTTP response > > Those discussions made me think of this possibility where my app > might be improperly implemented. I'm going to check the application > code from this perspective on Monday. When you use RECYCLE_FACADES, your application will only retain a reference to a useless object.... one which will likely NPE at some point when your code calls it. This makes the application safer (because one misbehaving servlet won't damage requests/responses currently in-use by subsequent requests) and also helps you find the error in your own code (because the problem will usually manifest itself as an earlier NPE instead of some "weirdness" which cannot really be readily explained because of the complexities of how objects are re-used, etc.). Hope that helps, - -chris -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlwFeG4ACgkQHPApP6U8 pFgxWxAAmG1UgpXnQUf2WdI6KQrgMa+Xx6nCkrr5XU/QK2EHvGO1jAE6Oc/qBmh4 ak25KzQJbC25BDCar/JdFuUlxAVwt2qht7M3RoCVqKm/BoZuamnnVXO5fq8a15cW NWMXFm9Unmh2wJGI12U8ZuuaPwte0rs4MSn+84gIzI81iaPmQB2TD6ohrl4mtJAT LsZQQks0IJKpu6/kpx2z0DPcqRTIynRIeegBzkvUjXOHFzu5wF99ZnS2etsTEbr6 bqKmEiSHSp9OjfiwggWMiHmnIPuZNwXziNj8cPpMxTWGLwMNXP1LUe7JSQ+UTKj4 tF5HYZovYW95h9vRHV/SkiJULmzi2nR/TGCe6noz0cNYon3y7kQJdG3i4GPaQ+bs +aww7homCjIceWREQbUKIp+/WUNXmtN8f8EwRjq94Av73CTi5inT+gv377D0Y6fG ui/p2vxkENsRDn/E6khTywxYjxOTruT0cmeYJ4Ee45k4EqMTb7142fxwblq1sgUy Of3ziwYVW7i6gz5zJaxv6oHxYCKVlHwj8aaqaETon5clGE7yFrRyAsilsLhu86OO 0ZwFLGUw7JSsiNs/KPODhmsY7XhNevmUu7ejhyfPe/TMle+TQyUmjTrdnjc+yX3C NwNUnZQUoQhkCseMaC2tp+6I1fo4/iFBLYdfCTrZpffQy4L0Okg= =uLmM -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
