On Wed, Jul 17, 2019 at 3:19 PM Mozgo Andrei <andrei.mo...@gmail.com> wrote:
> Hi all. > > Working on Update Tomcat 8.0.30 to Tomcat 9.0.14 I faced with a memory > leak issue. The application failed after some hours with the next > exception: > > > > *Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded* > > > The memory was completely consumed and the garbage collector was > struggling all the time trying to free it: > [image: 1.png] > > > Heap Dump contained 120 instances of > org.apache.catalina.connector.Response each consumed 10Mb of memory: > [image: image.png] > > I found out that the next Tomcat version 9.0.16 contains a fix for a bunch > of memory leak issues. Upgrade to 9.0.16 just doubled hour application > worked then it fails with the same issue. However, it could just be a > temporary load decrease. > > > > I tried to use Tomcat 8.5.16 that works fine with another module. Another > module heap dump didn't contain any instances of > org.apache.catalina.connector.Response even after more than 116 hours. > However, Tomcat 8.5.16 didn’t solve the problem. The application failed > with the same error and heap dump looked the same. > > > Then I looked into the heap dump of the application working on Tomcat > 8.0.30. It contained instances of org.apache.catalina.connector.Response as > well. But their size was much smaller: > > > [image: image.png] > > It looks like there is a memory leak issue with > org.apache.catalina.connector.Response in all Tomcat versions (8, 8.5, 9) > with the only difference in the size of OutputBuffer which is a part of > Response: > [image: image.png] > Here are GC roots of Response instance: > [image: image.png] > > Could you please give a piece of advice on how to manage with this? > First, having Response and OutputBuffer instances is normal. The OutputBuffer implementation was changed from using our own Byte/CharChunk impl to using Byte/CharBuffer. This is "ok" but has some side effects on memory use. In particular if response.setBufferSize is used. If it is, while previously it set an upper limit to which the buffer would gradually grow as needed, now it simply has to allocate the whole buffer. You might want to look into that. Also, the previous code used to discard the allocated char array, while now it is not possible. So now you have to be careful with use of setBufferSize (which is ok since using it is a very bad idea, the default is fine as it is and will perform well). Rémy > > Thanks, > Andrei Mozgo >
