Hi Chris, Here's what we have completed since the last post:
1. We have enabled AWTThreadProtection in the server.xml file by setting the attribute to true. 2. We have added a flush/close to the ImageIO code in our web application. We are still receiving the java.lang.IllegalStateException error. Can you go into more detail on the wrapper we need for the output stream? I pasted the error log below: Jun 6, 2013 10:35:05 AM org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet [jsp] in context with path [] threw exception [java.lang.IllegalStateException: getOutputStream() has already been called for this response] with root cause java.lang.IllegalStateException: getOutputStream() has already been called for this response at org.apache.catalina.connector.Response.getWriter(Response.java:639) at org.apache.catalina.connector.ResponseFacade.getWriter(ResponseFacade.java:214) at org.apache.jasper.runtime.JspWriterImpl.initOut(JspWriterImpl.java:125) at org.apache.jasper.runtime.JspWriterImpl.flushBuffer(JspWriterImpl.java:118) at org.apache.jasper.runtime.PageContextImpl.release(PageContextImpl.java:190) at org.apache.jasper.runtime.JspFactoryImpl.internalReleasePageContext(JspFactoryImpl.java:126) at org.apache.jasper.runtime.JspFactoryImpl.releasePageContext(JspFactoryImpl.java:80) at org.apache.jsp.captcha_jsp._jspService(captcha_jsp.java:233) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1008) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) -------------- The Tomcat service still is stopping randomly as well and generating the hs_err_pid*.log file upon stopping the apache tomcat service. # # A fatal error has been detected by the Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000001800057b2, pid=5768, tid=5708 # # JRE version: 6.0_21-b07 # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b17 mixed mode windows-amd64 ) # Problematic frame: # C [tcnative-1.dll+0x57b2] # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # --------------- T H R E A D --------------- Current thread (0x0000000004f32800): JavaThread "Finalizer" daemon [_thread_in_native, id=5708, stack(0x0000000005b50000,0x0000000005c50000)] siginfo: ExceptionCode=0xc0000005, reading address 0x0000000000000040 Registers: EAX=0x0000000000000000, EBX=0x0000000000000000, ECX=0x0000000007d560f0, EDX=0x0000000006468000 ESP=0x0000000005c4ee30, EBP=0x0000000000000000, ESI=0x0000000006b11240, EDI=0x000000000000006f EIP=0x00000001800057b2, EFLAGS=0x0000000000010206 Top of Stack: (sp=0x0000000005c4ee30) 0x0000000005c4ee30: 000000000000006f 0000000000000000 0x0000000005c4ee40: 0000000000000087 0000000000000000 0x0000000005c4ee50: 00000000013959ce 00000000013a1260 0x0000000005c4ee60: 0000000005c4eee0 000000000000006f 0x0000000005c4ee70: 0000000080d8bd40 0000000000000054 0x0000000005c4ee80: 000000000000006f 000000000000006f 0x0000000005c4ee90: 0000000085ad21d0 0000000005c4ee98 0x0000000005c4eea0: 0000000005c4eea0 0000000000000000 0x0000000005c4eeb0: 0000000005c4ef18 0000000080d8d358 0x0000000005c4eec0: 0000000000000000 0000000080d8bd40 0x0000000005c4eed0: 0000000000000000 0000000005c4ef00 0x0000000005c4eee0: 0000000005c4ef60 00000000013959ce 0x0000000005c4eef0: 0000000080d8d2a0 000000000139e316 0x0000000005c4ef00: 000000000000006f 0000000000000000 0x0000000005c4ef10: 0000000006b11240 0000000085ad21d0 0x0000000005c4ef20: 0000000005c4ef20 0000000081107f9e Instructions: (pc=0x00000001800057b2) 0x00000001800057a2: 8d 44 24 38 48 89 44 24 38 48 8b 46 30 48 03 d3 0x00000001800057b2: ff 50 40 85 c0 75 29 48 8b 44 24 38 48 85 c0 74 Stack: [0x0000000005b50000,0x0000000005c50000], sp=0x0000000005c4ee30, free space=3fb0000000000000000k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [tcnative-1.dll+0x57b2] [error occurred during error reporting (printing native stack), id 0xc0000005] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j org.apache.tomcat.jni.Socket.sendbb(JII)I+0 j org.apache.coyote.http11.InternalAprOutputBuffer.flushBuffer()V+22 j org.apache.coyote.http11.InternalAprOutputBuffer.flush()V+5 j org.apache.coyote.http11.AbstractHttp11Processor.action(Lorg/apache/coyote/ActionCode;Ljava/lang/Object;)V+130 j org.apache.coyote.Response.action(Lorg/apache/coyote/ActionCode;Ljava/lang/Object;)V+31 j org.apache.catalina.connector.OutputBuffer.doFlush(Z)V+97 j org.apache.catalina.connector.OutputBuffer.flush()V+2 j org.apache.catalina.connector.CoyoteOutputStream.flush()V+4 j javax.imageio.stream.FileCacheImageOutputStream.close()V+60 j javax.imageio.stream.ImageInputStreamImpl.finalize()V+8 v ~StubRoutines::call_stub j java.lang.ref.Finalizer.invokeFinalizeMethod(Ljava/lang/Object;)V+0 j java.lang.ref.Finalizer.runFinalizer()V+45 j java.lang.ref.Finalizer.access$100(Ljava/lang/ref/Finalizer;)V+1 j java.lang.ref.Finalizer$FinalizerThread.run()V+11 v ~StubRoutines::call_stub --------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x00000000064f4000 JavaThread "http-apr-80-exec-10" daemon [_thread_blocked, id=5400, stack(0x000000000df50000,0x000000000e050000)] 0x00000000064f3000 JavaThread "http-apr-80-exec-9" daemon [_thread_blocked, id=4544, stack(0x000000000de50000,0x000000000df50000)] 0x00000000064f2800 JavaThread "http-apr-80-exec-8" daemon [_thread_blocked, id=3584, stack(0x000000000dd50000,0x000000000de50000)] 0x00000000064f1800 JavaThread "http-apr-80-exec-7" daemon [_thread_blocked, id=1620, stack(0x000000000dc50000,0x000000000dd50000)] 0x00000000064f1000 JavaThread "http-apr-80-exec-6" daemon [_thread_blocked, id=4304, stack(0x000000000db50000,0x000000000dc50000)] 0x00000000064ef800 JavaThread "http-apr-80-exec-5" daemon [_thread_blocked, id=3440, stack(0x000000000da50000,0x000000000db50000)] 0x00000000064ef000 JavaThread "http-apr-80-exec-4" daemon [_thread_blocked, id=2180, stack(0x000000000d950000,0x000000000da50000)] 0x00000000064ee000 JavaThread "http-apr-443-exec-25" daemon [_thread_blocked, id=5284, stack(0x000000000d850000,0x000000000d950000)] 0x00000000064ed800 JavaThread "http-apr-443-exec-24" daemon [_thread_blocked, id=5300, stack(0x000000000d750000,0x000000000d850000)] 0x00000000064ec800 JavaThread "http-apr-443-exec-23" daemon [_thread_blocked, id=5296, stack(0x000000000d650000,0x000000000d750000)] 0x00000000064ec000 JavaThread "http-apr-443-exec-22" daemon [_thread_blocked, id=5292, stack(0x000000000d550000,0x000000000d650000)] 0x00000000064eb000 JavaThread "http-apr-443-exec-21" daemon [_thread_blocked, id=4412, stack(0x000000000d450000,0x000000000d550000)] 0x00000000078a3000 JavaThread "http-apr-443-exec-20" daemon [_thread_blocked, id=5316, stack(0x000000000d350000,0x000000000d450000)] 0x00000000078a2800 JavaThread "http-apr-443-exec-19" daemon [_thread_blocked, id=2880, stack(0x000000000d250000,0x000000000d350000)] 0x00000000078a2000 JavaThread "http-apr-443-exec-18" daemon [_thread_blocked, id=5324, stack(0x000000000d150000,0x000000000d250000)] 0x00000000078a1000 JavaThread "http-apr-443-exec-17" daemon [_thread_blocked, id=5192, stack(0x000000000d050000,0x000000000d150000)] 0x00000000078a0800 JavaThread "http-apr-443-exec-16" daemon [_thread_blocked, id=5276, stack(0x000000000cf50000,0x000000000d050000)] 0x000000000789f800 JavaThread "http-apr-443-exec-15" daemon [_thread_blocked, id=5244, stack(0x000000000ce50000,0x000000000cf50000)] 0x000000000789f000 JavaThread "http-apr-443-exec-14" daemon [_thread_blocked, id=5272, stack(0x000000000cd50000,0x000000000ce50000)] 0x000000000789e000 JavaThread "http-apr-443-exec-13" daemon [_thread_blocked, id=5280, stack(0x000000000cc50000,0x000000000cd50000)] 0x000000000789d800 JavaThread "http-apr-443-exec-12" daemon [_thread_blocked, id=5240, stack(0x000000000cb50000,0x000000000cc50000)] 0x000000000789c800 JavaThread "http-apr-443-exec-11" daemon [_thread_blocked, id=5236, stack(0x000000000ca50000,0x000000000cb50000)] 0x000000000789c000 JavaThread "http-apr-443-exec-10" daemon [_thread_blocked, id=5304, stack(0x000000000c950000,0x000000000ca50000)] 0x000000000789b800 JavaThread "http-apr-443-exec-9" daemon [_thread_in_native, id=5288, stack(0x000000000c850000,0x000000000c950000)] 0x000000000789a800 JavaThread "http-apr-443-exec-8" daemon [_thread_blocked, id=5196, stack(0x000000000c750000,0x000000000c850000)] 0x000000000789a000 JavaThread "http-apr-443-exec-7" daemon [_thread_blocked, id=5144, stack(0x000000000c650000,0x000000000c750000)] 0x0000000007899000 JavaThread "http-apr-443-exec-6" daemon [_thread_blocked, id=5216, stack(0x000000000c550000,0x000000000c650000)] 0x0000000007898800 JavaThread "http-apr-443-exec-5" daemon [_thread_blocked, id=4416, stack(0x000000000c450000,0x000000000c550000)] 0x0000000007897800 JavaThread "http-apr-443-exec-4" daemon [_thread_blocked, id=4396, stack(0x000000000c350000,0x000000000c450000)] 0x0000000007897000 JavaThread "http-apr-80-exec-3" daemon [_thread_blocked, id=5220, stack(0x000000000c250000,0x000000000c350000)] 0x0000000007896000 JavaThread "http-apr-80-exec-2" daemon [_thread_blocked, id=4656, stack(0x000000000c150000,0x000000000c250000)] 0x0000000007895800 JavaThread "http-apr-80-exec-1" daemon [_thread_blocked, id=5248, stack(0x000000000c050000,0x000000000c150000)] 0x0000000007895000 JavaThread "http-apr-443-exec-3" daemon [_thread_blocked, id=5132, stack(0x00000000086d0000,0x00000000087d0000)] 0x0000000007894000 JavaThread "http-apr-443-exec-2" daemon [_thread_blocked, id=4608, stack(0x00000000085d0000,0x00000000086d0000)] 0x0000000007d4f800 JavaThread "http-apr-443-exec-1" daemon [_thread_blocked, id=3236, stack(0x0000000008350000,0x0000000008450000)] 0x0000000007d4e800 JavaThread "ajp-apr-8009-AsyncTimeout" daemon [_thread_blocked, id=3660, stack(0x000000000bf50000,0x000000000c050000)] 0x0000000007d4e000 JavaThread "ajp-apr-8009-Acceptor-0" daemon [_thread_in_native, id=4224, stack(0x000000000be50000,0x000000000bf50000)] 0x0000000007d4d800 JavaThread "ajp-apr-8009-CometPoller-7" daemon [_thread_blocked, id=4056, stack(0x000000000bd50000,0x000000000be50000)] 0x0000000007d4c800 JavaThread "ajp-apr-8009-CometPoller-6" daemon [_thread_blocked, id=2664, stack(0x000000000bc50000,0x000000000bd50000)] 0x0000000007d4c000 JavaThread "ajp-apr-8009-CometPoller-5" daemon [_thread_blocked, id=5776, stack(0x000000000bb50000,0x000000000bc50000)] 0x0000000007d4b000 JavaThread "ajp-apr-8009-CometPoller-4" daemon [_thread_blocked, id=3148, stack(0x000000000ba50000,0x000000000bb50000)] 0x0000000007d4a800 JavaThread "ajp-apr-8009-CometPoller-3" daemon [_thread_blocked, id=368, stack(0x000000000b950000,0x000000000ba50000)] 0x0000000007d49800 JavaThread "ajp-apr-8009-CometPoller-2" daemon [_thread_blocked, id=6024, stack(0x000000000b850000,0x000000000b950000)] 0x0000000007d49000 JavaThread "ajp-apr-8009-CometPoller-1" daemon [_thread_blocked, id=5008, stack(0x000000000b750000,0x000000000b850000)] 0x0000000007d48000 JavaThread "ajp-apr-8009-CometPoller-0" daemon [_thread_blocked, id=3776, stack(0x000000000b650000,0x000000000b750000)] 0x0000000007d47800 JavaThread "ajp-apr-8009-Poller-7" daemon [_thread_blocked, id=348, stack(0x000000000b550000,0x000000000b650000)] 0x0000000007d47000 JavaThread "ajp-apr-8009-Poller-6" daemon [_thread_blocked, id=3592, stack(0x000000000b450000,0x000000000b550000)] 0x0000000007d46000 JavaThread "ajp-apr-8009-Poller-5" daemon [_thread_blocked, id=2204, stack(0x000000000b350000,0x000000000b450000)] 0x0000000007d45800 JavaThread "ajp-apr-8009-Poller-4" daemon [_thread_blocked, id=816, stack(0x000000000b250000,0x000000000b350000)] 0x0000000007d44800 JavaThread "ajp-apr-8009-Poller-3" daemon [_thread_blocked, id=1728, stack(0x000000000b150000,0x000000000b250000)] 0x0000000007d44000 JavaThread "ajp-apr-8009-Poller-2" daemon [_thread_blocked, id=3588, stack(0x000000000b050000,0x000000000b150000)] 0x0000000007d43000 JavaThread "ajp-apr-8009-Poller-1" daemon [_thread_blocked, id=4324, stack(0x000000000af50000,0x000000000b050000)] 0x0000000007d42800 JavaThread "ajp-apr-8009-Poller-0" daemon [_thread_blocked, id=3208, stack(0x000000000ae50000,0x000000000af50000)] 0x0000000007d41800 JavaThread "http-apr-443-AsyncTimeout" daemon [_thread_blocked, id=5000, stack(0x000000000ad50000,0x000000000ae50000)] 0x0000000007d41000 JavaThread "http-apr-443-Acceptor-0" daemon [_thread_in_native, id=2528, stack(0x000000000ac50000,0x000000000ad50000)] 0x0000000007d40800 JavaThread "http-apr-443-CometPoller-7" daemon [_thread_blocked, id=252, stack(0x000000000ab50000,0x000000000ac50000)] 0x0000000006f60000 JavaThread "http-apr-443-CometPoller-6" daemon [_thread_blocked, id=1628, stack(0x000000000aa50000,0x000000000ab50000)] 0x0000000006f5f800 JavaThread "http-apr-443-CometPoller-5" daemon [_thread_blocked, id=300, stack(0x000000000a950000,0x000000000aa50000)] 0x0000000006f5f000 JavaThread "http-apr-443-CometPoller-4" daemon [_thread_blocked, id=2580, stack(0x000000000a850000,0x000000000a950000)] 0x0000000006f5e000 JavaThread "http-apr-443-CometPoller-3" daemon [_thread_blocked, id=3984, stack(0x000000000a750000,0x000000000a850000)] 0x0000000006f5d800 JavaThread "http-apr-443-CometPoller-2" daemon [_thread_blocked, id=2144, stack(0x000000000a650000,0x000000000a750000)] 0x0000000006f5c800 JavaThread "http-apr-443-CometPoller-1" daemon [_thread_blocked, id=2188, stack(0x000000000a550000,0x000000000a650000)] 0x0000000006f5c000 JavaThread "http-apr-443-CometPoller-0" daemon [_thread_blocked, id=3212, stack(0x000000000a450000,0x000000000a550000)] 0x0000000006f5b000 JavaThread "http-apr-443-Poller-7" daemon [_thread_blocked, id=5992, stack(0x000000000a350000,0x000000000a450000)] 0x0000000006f5a800 JavaThread "http-apr-443-Poller-6" daemon [_thread_blocked, id=248, stack(0x000000000a250000,0x000000000a350000)] 0x0000000006f59800 JavaThread "http-apr-443-Poller-5" daemon [_thread_blocked, id=3788, stack(0x000000000a150000,0x000000000a250000)] 0x0000000006f59000 JavaThread "http-apr-443-Poller-4" daemon [_thread_blocked, id=4736, stack(0x000000000a050000,0x000000000a150000)] 0x0000000006f58800 JavaThread "http-apr-443-Poller-3" daemon [_thread_blocked, id=2228, stack(0x0000000009f50000,0x000000000a050000)] 0x0000000006f57800 JavaThread "http-apr-443-Poller-2" daemon [_thread_blocked, id=5696, stack(0x0000000009e50000,0x0000000009f50000)] 0x0000000006f57000 JavaThread "http-apr-443-Poller-1" daemon [_thread_blocked, id=2108, stack(0x0000000009d50000,0x0000000009e50000)] 0x0000000006f56000 JavaThread "http-apr-443-Poller-0" daemon [_thread_blocked, id=4696, stack(0x0000000009c50000,0x0000000009d50000)] 0x0000000006f55800 JavaThread "http-apr-80-AsyncTimeout" daemon [_thread_blocked, id=2620, stack(0x0000000009b50000,0x0000000009c50000)] 0x0000000006f54800 JavaThread "http-apr-80-Acceptor-0" daemon [_thread_in_native, id=2736, stack(0x0000000009a50000,0x0000000009b50000)] 0x0000000006f54000 JavaThread "http-apr-80-Sendfile-0" daemon [_thread_blocked, id=2488, stack(0x0000000009950000,0x0000000009a50000)] 0x0000000006f53000 JavaThread "http-apr-80-CometPoller-7" daemon [_thread_blocked, id=4640, stack(0x0000000009850000,0x0000000009950000)] 0x0000000006f52800 JavaThread "http-apr-80-CometPoller-6" daemon [_thread_blocked, id=2900, stack(0x0000000009750000,0x0000000009850000)] 0x0000000006f52000 JavaThread "http-apr-80-CometPoller-5" daemon [_thread_blocked, id=4904, stack(0x0000000009650000,0x0000000009750000)] 0x0000000006f51000 JavaThread "http-apr-80-CometPoller-4" daemon [_thread_blocked, id=3160, stack(0x0000000009550000,0x0000000009650000)] 0x0000000006ca3800 JavaThread "http-apr-80-CometPoller-3" daemon [_thread_blocked, id=3580, stack(0x0000000009450000,0x0000000009550000)] 0x0000000006ca3000 JavaThread "http-apr-80-CometPoller-2" daemon [_thread_blocked, id=2600, stack(0x0000000009350000,0x0000000009450000)] 0x0000000006ca2000 JavaThread "http-apr-80-CometPoller-1" daemon [_thread_blocked, id=3192, stack(0x0000000009250000,0x0000000009350000)] 0x0000000006ca1800 JavaThread "http-apr-80-CometPoller-0" daemon [_thread_blocked, id=460, stack(0x0000000009150000,0x0000000009250000)] 0x0000000006ca0800 JavaThread "http-apr-80-Poller-7" daemon [_thread_blocked, id=3168, stack(0x0000000009050000,0x0000000009150000)] 0x0000000006ca0000 JavaThread "http-apr-80-Poller-6" daemon [_thread_blocked, id=4760, stack(0x0000000008f50000,0x0000000009050000)] 0x0000000006c9f000 JavaThread "http-apr-80-Poller-5" daemon [_thread_blocked, id=6100, stack(0x0000000008e50000,0x0000000008f50000)] 0x0000000006c9e800 JavaThread "http-apr-80-Poller-4" daemon [_thread_blocked, id=6104, stack(0x0000000008d50000,0x0000000008e50000)] 0x0000000006c9d800 JavaThread "http-apr-80-Poller-3" daemon [_thread_blocked, id=2564, stack(0x0000000008c50000,0x0000000008d50000)] 0x0000000006c9d000 JavaThread "http-apr-80-Poller-2" daemon [_thread_blocked, id=1276, stack(0x0000000008b50000,0x0000000008c50000)] 0x0000000006c9c800 JavaThread "http-apr-80-Poller-1" daemon [_thread_blocked, id=5568, stack(0x0000000008a50000,0x0000000008b50000)] 0x0000000007176800 JavaThread "http-apr-80-Poller-0" daemon [_thread_blocked, id=5820, stack(0x0000000008950000,0x0000000008a50000)] 0x000000000779d800 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=4484, stack(0x0000000008850000,0x0000000008950000)] 0x00000000064ab000 JavaThread "AWT-Windows" daemon [_thread_in_native, id=6064, stack(0x0000000008450000,0x0000000008550000)] 0x000000000677f800 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=6124, stack(0x0000000008250000,0x0000000008350000)] 0x00000000064ea800 JavaThread "GC Daemon" daemon [_thread_blocked, id=5712, stack(0x0000000007e50000,0x0000000007f50000)] 0x0000000004faa000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=6132, stack(0x0000000006050000,0x0000000006150000)] 0x0000000004fa0800 JavaThread "CompilerThread1" daemon [_thread_blocked, id=6016, stack(0x0000000005f50000,0x0000000006050000)] 0x0000000004f9a800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=6112, stack(0x0000000005e50000,0x0000000005f50000)] 0x0000000004f99800 JavaThread "Attach Listener" daemon [_thread_blocked, id=4456, stack(0x0000000005d50000,0x0000000005e50000)] 0x0000000004f99000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6060, stack(0x0000000005c50000,0x0000000005d50000)] =>0x0000000004f32800 JavaThread "Finalizer" daemon [_thread_in_native, id=5708, stack(0x0000000005b50000,0x0000000005c50000)] 0x0000000004f30000 JavaThread "Reference Handler" daemon [_thread_blocked, id=1280, stack(0x0000000005a50000,0x0000000005b50000)] 0x00000000003c8800 JavaThread "main" [_thread_in_native, id=6088, stack(0x0000000000bb0000,0x0000000000cb0000)] Other Threads: 0x0000000004f2d000 VMThread [stack: 0x0000000005950000,0x0000000005a50000] [id=5396] 0x0000000004fc0000 WatcherThread [stack: 0x0000000006150000,0x0000000006250000] [id=6140] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None Heap PSYoungGen total 50048K, used 1911K [0x00000000da950000, 0x00000000decb0000, 0x00000001053f0000) eden space 33536K, 5% used [0x00000000da950000,0x00000000dab2dc40,0x00000000dca10000) from space 16512K, 0% used [0x00000000ddc90000,0x00000000ddc90000,0x00000000decb0000) to space 17728K, 0% used [0x00000000dca10000,0x00000000dca10000,0x00000000ddb60000) PSOldGen total 71744K, used 27131K [0x00000000853f0000, 0x0000000089a00000, 0x00000000da950000) object space 71744K, 37% used [0x00000000853f0000,0x0000000086e6eea8,0x0000000089a00000) PSPermGen total 51072K, used 25391K [0x000000007fff0000, 0x00000000831d0000, 0x00000000853f0000) object space 51072K, 49% used [0x000000007fff0000,0x00000000818bbd88,0x00000000831d0000) Dynamic libraries: 0x0000000140000000 - 0x000000014001e000 C:\Program Files\Apache Software Foundation\Tomcat 5.5\bin\Tomcat7.exe 0x0000000077ec0000 - 0x0000000077ffb000 C:\WINDOWS\system32\ntdll.dll 0x0000000077d40000 - 0x0000000077eb3000 C:\WINDOWS\system32\kernel32.dll 0x000007ff7fee0000 - 0x000007ff7ffe6000 C:\WINDOWS\system32\ADVAPI32.dll 0x000007ff7fd30000 - 0x000007ff7fec9000 C:\WINDOWS\system32\RPCRT4.dll 0x000007ff7e9c0000 - 0x000007ff7e9e2000 C:\WINDOWS\system32\Secur32.dll 0x000007ff7f190000 - 0x000007ff7fb9b000 C:\WINDOWS\system32\SHELL32.dll 0x000007ff7fc00000 - 0x000007ff7fc86000 C:\WINDOWS\system32\msvcrt.dll 0x000007ff7fc90000 - 0x000007ff7fd2c000 C:\WINDOWS\system32\GDI32.dll 0x0000000077c20000 - 0x0000000077d2c000 C:\WINDOWS\system32\USER32.dll 0x000007ff7ef60000 - 0x000007ff7effb000 C:\WINDOWS\system32\SHLWAPI.dll 0x000007ff7d500000 - 0x000007ff7d539000 C:\WINDOWS\system32\IMM32.DLL 0x000000006fa00000 - 0x000000006fa3f000 C:\PROGRA~2\Sophos\SOPHOS~1\SOPHOS~2.DLL 0x000007ff7e380000 - 0x000007ff7e390000 C:\WINDOWS\system32\PSAPI.DLL 0x000007ff7f000000 - 0x000007ff7f187000 C:\WINDOWS\WinSxS\amd64_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.3790.3959_x-ww_0A7B2435\comctl32.dll 0x000000006d890000 - 0x000000006df61000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\server\jvm.dll 0x000007ff7e240000 - 0x000007ff7e290000 C:\WINDOWS\system32\WINMM.dll 0x000000006d800000 - 0x000000006d80e000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\verify.dll 0x000000006d450000 - 0x000000006d477000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\java.dll 0x000000006d3b0000 - 0x000000006d3ba000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\hpi.dll 0x000000006d850000 - 0x000000006d862000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\zip.dll 0x000000006d5e0000 - 0x000000006d5ea000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\management.dll 0x0000000180000000 - 0x000000018018c000 C:\Program Files\Apache Software Foundation\Tomcat 5.5\bin\tcnative-1.dll 0x000007ff77310000 - 0x000007ff77340000 C:\WINDOWS\system32\WS2_32.dll 0x000007ff77140000 - 0x000007ff7714c000 C:\WINDOWS\system32\WS2HELP.dll 0x000007ff771b0000 - 0x000007ff7722d000 C:\WINDOWS\system32\MSWSOCK.dll 0x0000000068000000 - 0x000000006804c000 C:\WINDOWS\system32\rsaenh.dll 0x000000006d0a0000 - 0x000000006d263000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\awt.dll 0x000007ff78f10000 - 0x000007ff78f51000 C:\WINDOWS\system32\WINSPOOL.DRV 0x000007ff57140000 - 0x000007ff573c5000 C:\WINDOWS\system32\ole32.dll 0x000007ff6d200000 - 0x000007ff6d293000 C:\WINDOWS\system32\hnetcfg.dll 0x000007ff77170000 - 0x000007ff7717b000 C:\WINDOWS\System32\wshtcpip.dll 0x000000006d6a0000 - 0x000000006d6b7000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\net.dll 0x000007ff7e8f0000 - 0x000007ff7e93e000 C:\WINDOWS\system32\DNSAPI.dll 0x000007ff7e9f0000 - 0x000007ff7e9fb000 C:\WINDOWS\System32\winrnr.dll 0x000007ff7e950000 - 0x000007ff7e9b6000 C:\WINDOWS\system32\WLDAP32.dll 0x000007ff7ea00000 - 0x000007ff7ea07000 C:\WINDOWS\system32\rasadhlp.dll 0x000000006d340000 - 0x000000006d3ab000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\fontmanager.dll 0x000000006d6c0000 - 0x000000006d6cb000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\nio.dll 0x000000006d570000 - 0x000000006d5a5000 C:\Program Files\Java\jdk1.6.0_21\jre\bin\jpeg.dll VM Arguments: jvm_args: -Dcatalina.home=C:\Program Files\Apache Software Foundation\Tomcat 5.5 -Dcatalina.base=C:\Program Files\Apache Software Foundation\Tomcat 5.5 -Djava.endorsed.dirs=C:\Program Files\Apache Software Foundation\Tomcat 5.5\endorsed -Djava.io.tmpdir=C:\Program Files\Apache Software Foundation\Tomcat 5.5\temp -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=C:\Program Files\Apache Software Foundation\Tomcat 5.5\conf\logging.properties exit java_command: <unknown> Launcher Type: generic Environment Variables: PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files (x86)\Dell\SysMgt\RAC5;C:\Program Files (x86)\Dell\SysMgt\oma\bin;C:\WINDOWS\system32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Common Files\Acronis\SnapAPI\ OS=Windows_NT PROCESSOR_IDENTIFIER=EM64T Family 6 Model 23 Stepping 10, GenuineIntel --------------- S Y S T E M --------------- OS: Windows Server 2003 family Build 3790 Service Pack 2 CPU:total 8 (4 cores per cpu, 1 threads per core) family 6 model 23 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1 Memory: 4k page, physical 8386584k(6392804k free), swap 12121980k(10896560k free) vm_info: Java HotSpot(TM) 64-Bit Server VM (17.0-b17) for windows-amd64 JRE (1.6.0_21-b07), built on Jul 17 2010 01:05:36 by "java_re" with MS VC++ 8.0 (VS2005) time: Thu Jun 06 10:04:05 2013 elapsed time: 1386 seconds -----Original Message----- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Sent: Tuesday, May 21, 2013 2:49 PM To: Tomcat Users List Subject: Re: Apache Tomcat service has been shutting down/stopping randomly. -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 James, On 5/21/13 1:25 PM, James Snider wrote: > Below is the most recent Exception Access Violation tcnative Error we > are receiving. This error log is generated every time the service > randomly stops causing us to believe this to be the primary catalyst. So now the big question: can you reliably trigger the "primary catalyst"? If so, we can try a few things. First, figure out how to blow-up your application. That shouldn't be that tough: just make sure you exercise the code that uses ImageIO to write directly to a ServletOutputStream. Since you said that you aren't using ImageIO, this might present a challenge.... but you are going to have to find it. Next, implement the OutputStream-wrapper[1] in the code that bombs. Then, try to bomb your webapp again. If it no longer bombs, then you are probably done: congratulations. If it still bombs, you might have missed a use of ImageIO: look again, re-wrap and re-try. If that does not work, we could use some more information about your environment. IIRC, you have installed Tomcat 7.0.40 including the bundled tcnative library, on Microsoft Windows in a directory called "Tomcat 5.5" because of some hard-coded paths. So, everything is TC7.0.40, the library is statically-linked, and the tcnative version is 1.2.27. Can you confirm the above and also confirm the versions of tcnative, APR, and OpenSSL you have? They should all be displayed in your catalina.out log file (or wherever stdout goes when you launch Tomcat) near the beginning of the output after a fresh TC launch. If you are in production and just want to stop crashing, then you should switch connectors: use either the BIO or NIO connectors and you should avoid the fatal crash that occurs when the native stream has (likely) been closed and nulled-out. - -chris [1] http://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRm8GxAAoJEBzwKT+lPKRYg6kP/3WHQ6+iWjqxo4VEhKio0vos fvY7YBu3LOGg0Yu0h1Flvl26JNo+iAW4fl+KsAetZVZQFX3GBV7mN4meKh5GuS60 ft3eu9fXL+i0oPQUyy4m4IK2RmuBbMSNp0f9q0MtifRpIOlyW4/XfcLesgiFsOxM dLHoWFYppDwW812x7zdjCzSZxlvNgVhPPnO9UipfM4oGMH5FSqTe/c19Omi7s4gv La2WpRSF2QDAQ5ldEXfblsOd2fSo8nDJEkJ7MwYb60/w6eEmxxJO2Y+NZfesA9wU HXXqUcNevU0CsiEsE4Ji05CzHPgNg+EGsaAYhjeZSVkdmBl6frm40SuKAe5c7rVc mRPgq5TyHnvr46ypar6vWoxaF4HVl0vNRgc1/jGE3y2WdCFBRm3K8wVWFGzrYaRd 1bbeyXxsEsxlkt8YvaL4/IXrvqOV9zhh1N9WuiR2lV5f4rWxwnBzX96J2VhA6/CI JyIeSl+f8G7Bm7H/Ey0Vxz97lq1iDWB6W+V4psKksXzgdghOh5TagnVg1mOW7ZC2 mGuCffu5bQcbW8hwmAZhMBbUujxGOupSAyrlBezFR2iCLOMrMQq38dmHW6juG440 APhCun1hiLENNAFsfmn3YvQXawG30hEs1jdZpJ91iJjHwQkIaKviJEvLkK1svMAt jFIkQ8854CUckhcWLYnS =Uu3w -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org ------------------ CONFIDENTIALITY NOTICE --------------- This message, including any attachments, is for the sole use of the intended recipient(s) and may contain privileged confidential information protected by law. Any unauthorized review, use, disclosure or distribution of this message is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of this message. ------------------ CONFIDENTIALITY NOTICE ---------------