Hi, I am in charge of running a Apache-2, Tomcat-7, Ubuntu-10.04 set up for which we have to be PCI Compliant. We recently upgraded to Apache-2.2.17 and Tomcat-7.0.8 (from Apache-2.0.x and Tomcat 5.0.28) in order to comply with the requirements of the PCI Compliance checks and ironed out any issues to get us back to a satisfactory running state.
We have now received warning, from our PCI Compliance monitor service, that further updates are required to remain compliant, those being to use Apache-2.2.19 and Tomcat-7.0.14 (or higher). When using Tomcat-7.0.8, we experience healthy memory cycling. The Old Generation slowly increases until garbage collection runs and clears out the Old Gen memory level, dropping considerably as expected. So to the upgrading! Upgrading Apache has been successful and without problem, however upgrading Tomcat has caused memory problems which, as yet, I cannot find similar reported cases or clear possibilities for why its happening. I've searched post archives and web results, looked through the change log of Tomcat and tried using tools like jmap and the YourKit monitor to discover some answers, although I am quite inexperienced with these tools, but unfortunately I'm not progressing so far. I have tried running with... - Tomcat-7.0.16 (being the latest available) - Tomcat-7.0.14 (being the minimum required for PCI Compliance) - Tomcat-7.0.10 (being the next available release after 7.0.8) Tomcat-7.0.10 obviously has the fewest number changes from Tomcat-7.0.8 but, all of these have resulted in seeing a rather sharp rise in the Old Generation memory usage, reaching 90%+ (of the available 1.25GB) in under an hour. Requesting forced garbage collection gives little effect, if not none at all, and eventually the Old Gen memory becomes full and the site stops functioning. There have been no changes to the jvm settings, so I doubt they are the cause, but for your information they are as follows... -Djava.awt.headless=true -Xmx1536m -Xms1536m -XX:NewSize=256m -XX:MaxNewSize=256M -XX:SurvivorRatio=6 -XX:ReservedCodeCacheSize=64m -XX:MaxPermSize=512m -XX:PermSize=512m I have used jmap to look into what objects are being held in memory for the problematic versions of Tomcat. For Tomcat-7.0.10, the top 20 listed entries are... num #instances #bytes class name ---------------------------------------------- 1: 8170940 471172552 [C 2: 8501813 272058016 java.lang.String 3: 5388341 215533640 javax.servlet.jsp.tagext.TagAttributeInfo 4: 581051 35164552 [Ljava.lang.Object; 5: 170658 34508384 <constMethodKlass> 6: 528746 33839744 javax.servlet.jsp.tagext.TagInfo 7: 528746 31094960 [Ljavax.servlet.jsp.tagext.TagAttributeInfo; 8: 75231 25469312 [B 9: 170658 23223888 <methodKlass> 10: 395025 22121400 org.apache.jasper.compiler.Mark 11: 11737 21889840 <constantPoolKlass> 12: 281224 20248128 org.apache.jasper.compiler.Node$TemplateText 13: 229740 19444304 [Ljava.util.HashMap$Entry; 14: 594062 19009984 java.util.HashMap$Entry 15: 220856 15973856 <symbolKlass> 16: 110747 13909808 [Ljava.lang.String; 17: 10561 13740840 <constantPoolCacheKlass> 18: 400243 12807776 java.util.Stack 19: 224826 10791648 java.util.HashMap 20: 11737 10042552 <instanceKlassKlass> When running under Tomcat-7.0.8 after approximately a similar time of about an hour, the top 20 entries are... num #instances #bytes class name ---------------------------------------------- 1: 592439 86974504 [C 2: 47888 73416688 [I 3: 189957 39457512 <constMethodKlass> 4: 88307 27806528 [B 5: 13444 26059552 <constantPoolKlass> 6: 189957 25848632 <methodKlass> 7: 619859 19835488 java.lang.String 8: 242174 19462792 <symbolKlass> 9: 12260 16256800 <constantPoolCacheKlass> 10: 13444 11624240 <instanceKlassKlass> 11: 236662 9466480 java.math.BigDecimal 12: 184843 7393720 javax.servlet.jsp.tagext.TagAttributeInfo 13: 83238 5795576 [Ljava.lang.Object; 14: 59168 5611728 [Ljava.util.HashMap$Entry; 15: 132465 4238880 java.util.HashMap$Entry 16: 5287 3630456 <methodDataKlass> 17: 54218 2602464 java.util.HashMap 18: 26369 2320472 java.lang.reflect.Method 19: 39714 1588560 java.util.TreeMap$Entry 20: 14143 1470872 java.lang.Class It seems that the character arrays [C, java.lang.String and javax.servlet.jsp.tagext.TagAttributeInfo entries are considerably higher in Tomcat-7.0.10 than in Tomcat-7.0.8 and I am wondering if this could lead to an explanation for the difference. Would anyone know of any changes between the two versions, possibly linked to those memory entries, that could lead to such behaviour? Any help or suggestions is greatly appreciated! I'm sorry for a long post, but hopefully its got the information needed to help diagnosis. Thanks in advance, Ian --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
