On 14/09/2011 08:34, Darius D. wrote: > > Hi, > > we have a problem of request params beeing null in servlets where it > shouldn't be ( post or get, in get we can even see that > request.getQueryString() is fine, but params are null). > > It is more pronounced when using dedicated thread executor, moving to > threads provided by connector itself (with maxThreads="500" option using NIO > ) seems to be lowering counts of errors. > > > Atm we are seeing 3 modes of failure in our tomcat setup ( nginx reverse > proxy -> tomcat with NIO connector ) > > > 1) > SEVERE: Error finishing response > java.lang.NullPointerException > > and/or > > SEVERE: Error processing request > java.lang.NullPointerException > > in logs. ( could be some app problem, but still shady, as no stack trace is > given, like if exception object is gone itself??? ) > > 2) Sometimes we get 502 (from nginx, that got upstream error) like Rakesh in > other thread, from servlets that should be fine ( result of NPE in real > strange places ). > > 3) And most scary ones are the following: > > 2011 09 13 14:18:20.568 WARN Servlet - error response: error=no action > query: [type=groupsChangedAction&v=1315912033175] content type:null > protocol: HTTP/1.0 length:-1 character encoding: null GET > 2011 09 13 14:18:20.937 WARN Servlet - error response: error=no action > query: [type=groupsChangedAction&v=1315912699092] content type:null > protocol: HTTP/1.0 length:-1 character encoding: null GET > > In tomcat access log they are shown as: > 127.0.0.1 - - [13/Sep/2011:14:18:20 +0300] "GET > /ourapp/servlet/Servlet?type=groupsChangedAction&v=1315912033175 HTTP/1.0" > 200 15 > 127.0.0.1 - - [13/Sep/2011:14:18:20 +0300] "GET > /ourapp/servlet/Servlet?type=groupsChangedAction&v=1315912699092 HTTP/1.0" > 200 15 > > So as you can see, tomcat got that GET reqest with type and v params like it > gets and processes thousands others. > > Servlet code looks like:
doProcess(req, res) is not a Servlet API method. What other method(s)
call it?
> private void doProcess(HttpServletRequest request, HttpServletResponse
> response) throws IOException {
> HttpSession session = request.getSession();
> if (session != null) {
>
> String action = request.getParameter("type");
> logger.debug("action: " + action);
>
> if (action == null) {
>
> printError(request, response,"error=no
> action");
> ....
>
> private void printError(HttpServletRequest request,
> HttpServletResponse response, String error) throws IOException {
> logger.warn("error response: " + error + " ip: " +
> request.getRemoteAddr() + " query: [" + request.getQueryString() + "]" + "
> content type:" + request.getContentType() + " protocol: " +
> request.getProtocol() + " length:" + request.getContentLength() + "
> character encoding: " + request.getCharacterEncoding() + " " +
> request.getMethod());
>
> response.setContentType("text/plain");
> PrintWriter out = response.getWriter();
> out.print(error);
> out.flush();
> out.close();
> }
>
> It happens several times in a day in bursts ( GC collecting something?).
>
>
> So really puzzling and random error, tomcat got proper request as shown in
> access logs (and even request.getQueryString() is valid :) ) , processed
> request from same user 5s ago, will process one more 5s later, but somehow
> request is NULL??? Now we handle it by retrying, but such random NULL is
> fatal for other servlets and services that throw NPEs and it seems that some
> tomcat code is affected too with (1).
>
> I'd grateful if some smart guys could look into this problem and willing to
> provide any other info.
Usually this type of thing occurs because the request (or response) is
being held as an instance field somewhere in a thread-unsafe way.
p
> Only connector running is:
> <Connector port="8080"
> protocol="org.apache.coyote.http11.Http11NioProtocol"
> connectionTimeout="20000"
> enableLookups="false"
> acceptCount="1000"
> maxThreads="500"
> maxPostSize="10485760"
> />
>
> Also using
> <Valve className="org.apache.catalina.valves.RemoteIpValve"
> protocolHeader="X-Forwarded-Proto" />
>
>
> BR,
>
> Darius.
>
>
>
>
signature.asc
Description: OpenPGP digital signature
