You can determine the anount of remaining RAM and then set MaxRows and FetchSize to the number of Rows e.g. http://www.docjar.com/docs/api/java/lang/Runtime.html //determine how much memory is free for allocating Obects on the heap long remainingRAM = java.lang.Runtime.getRuntime().freeMemory(); int RowsToFetch = remainingRAM/RowSize;
http://www.docjar.com/docs/api/java/sql/Statement.html Statement.setFetchSize(RowsToFetch); Statement.setMaxRows(RowsToFetch); Martin-- ----- Original Message ----- From: "A C" <[EMAIL PROTECTED]> To: "Tomcat Users List" <users@tomcat.apache.org> Sent: Friday, October 12, 2007 3:47 PM Subject: Re: placing jars in /server/lib - can cause oome? > har har. glad i could give everyone on this thread a good chuckle. > > anyway... I didn't want to rattle off more crud, so instead I collected as > best I could, information that might elucidate my issue more: > > > and everyone was right, my jar file did NOT live in \server. > > > Application: Reporting application we'll call "reporting.war" , which > unpacks to a directory size of ~ 100MB of jars/classes/static content/etc. > # webapp contexts (is that what these are called?): 40 > Server: Tomcat 5.0.27 > JDK: 1.4.2_13 > > Tomcat runs as a windows service: Tomcat\bin\tomcat5.exe > //RS//reportingTomcat > > -Xms: 512MB > -Xmx: 1024M > -Xrs > > ________________________________________ > > > > The only "thing" running on this machine is a Windows apache tomcat > service. this application "hooks" into a sister app server that performs > the "meat and potatos" activities of the application which we've never > needed to restart but have done so to see if it was somehow related to > web-layer OOMEs (hey, my father used to bang the t.v. set to make the > picture more clear and it worked for HIM, so ...) > > webapp/reporting/web-inf/lib = 93 jar files = 22 Mb > webapp/reporting01 > webapp/reporting02 > . > . > webapp/reporting30 > > each of these contexts has a corresponding config file in : > > \conf\Catalina\localhost\reporting01.xml > \conf\Catalina\localhost\reporting02.xml > . > . > \conf\Catalina\localhost\reporting30.xml > > In this XMl file, I have: > > <Context docBase="E:\tools\portal\reporting.war" path="/reporting01" > crossContext="false" debug="0" reloadable="false" trusted="false"> > <Logger className="org.apache.catalina.logger.FileLogger" > prefix="localhost_reporting01_log." suffix=".txt" timestamp="true"/> > <Resource name="jdbc/myapp" auth="Container" type="javax.sql.DataSource" /> > <ResourceParams name="jdbc/myapp"> > <parameter> > <name>factory</name> > <value>org.apache.commons.dbcp.BasicDataSourceFactory</value> > </parameter> > <parameter> > <name>driverClassName</name> > <value>net.sourceforge.jtds.jdbc.Driver</value> > </parameter> > <parameter> > <name>url</name> > <value>jdbc:jtds:sqlserver://sqlserver:1433/reporting01;User=xxx;Password=xx x</value> > <!-- to add DB instance, use "...:port/databaseName;instance=<name>;User=xx" > notation --> > </parameter> > <parameter> > <name>validationQuery</name> > <value>select 1</value> > </parameter> > <parameter> > <name>maxActive</name> > <value>30</value> > </parameter> > </ResourceParams> > </Context> > > > ________________________________________________ > > > The *only* customization we've done is to make a brief call to a database, > grab a valid userlogin from our application and perform a handshake to the > reporting application. > > _________ custom code lives in \webapps\reportingXX\customcode\ ________ > > and all apps use " jtds-1.2.jar " which lives in \Tomcat\common\lib > > our customization is something like: > > file > db.include, which has: > > InitialContext initCtx = new InitialContext(); > Context envCtx = (Context) initCtx.lookup("java:comp/env"); > DataSource ds = (DataSource) envCtx.lookup("jdbc/myapp"); > > ...and we create the connection in: > > file > login.jsp which has: > > includes db.include > > > Connection con = ds.getConnection; > > we do some SQL and then close/null any variables using or related to the > connection/resultset/etc > > > After about a day, we start seeing these: > > __________________________________________________ > > 2007-10-12 12:24:20 ApplicationDispatcher[/reportingXX] Servlet.service() > for servlet jsp threw exception > java.lang.OutOfMemoryError > > 2007-10-12 12:24:21 ApplicationDispatcher[[/reportingXX] ] Servlet.service() > for servlet jsp threw exception > java.lang.OutOfMemoryError > > 2007-10-12 12:24:21 StandardWrapperValve[action]: Servlet.service() for > servlet action threw exception > java.lang.OutOfMemoryError > > __________________________________________________ > > Here's the permgen after 1.5 hours of my simulation (35 deployed webapps of > this application) > > Start: > > 9.643: [GC 9.643: [DefNew: 46591K->3981K(52416K), 0.0273929 secs] > 46591K->3981K(518464K), 0.0276010 secs] > 13.569: [GC 13.569: [DefNew: 50573K->3714K(52416K), 0.0351330 secs] > 50573K->7132K(518464K), 0.0352592 secs] > 19.409: [GC 19.409: [DefNew: 50306K->4897K(52416K), 0.0262670 secs] > 53724K->9800K(518464K), 0.0263980 secs] > 22.330: [GC 22.330: [DefNew: 51489K->1120K(52416K), 0.0177011 secs] > 56392K->10751K(518464K), 0.0178319 secs] > 22.978: [GC 22.978: [DefNew: 47712K->1128K(52416K), 0.0031998 secs] > 57343K->10760K(518464K), 0.0033108 secs] > 24.976: [GC 24.976: [DefNew: 47720K->3289K(52416K), 0.0163705 secs] > 57352K->12920K(518464K), 0.0164999 secs] > 38.553: [GC 38.553: [DefNew: 49881K->5047K(52416K), 0.0225587 secs] > 59512K->15380K(518464K), 0.0227062 secs] > 71.451: [GC 71.451: [DefNew: 51635K->249K(52416K), 0.0224671 secs] > 61967K->13712K(518464K), 0.0225193 secs] > 97.070: [GC 97.070: [DefNew: 46841K->1806K(52416K), 0.0066059 secs] > 60304K->15268K(518464K), 0.0066603 secs] > 107.194: [GC 107.194: [DefNew: 48398K->4604K(52416K), 0.0193955 secs] > 61860K->18066K(518464K), 0.0194505 secs] > 136.409: [GC 136.409: [DefNew: 51195K->251K(52416K), 0.0164256 secs] > 64658K->16415K(518464K), 0.0164884 secs] > 169.849: [GC 169.849: [DefNew: 46843K->1755K(52416K), 0.0072353 secs] > 63007K->17919K(518464K), 0.0072906 secs] > . > . > . > . > . > . > > 1247.073: [GC 1247.073: [DefNew: 51055K->276K(52480K), 0.0630231 secs] > 123930K->75721K(518528K), 0.0630960 secs] > 1279.579: [GC 1279.579: [DefNew: 46932K->1585K(52480K), 0.0256284 secs] > 122377K->77031K(518528K), 0.0256938 secs] > 1291.668: [GC 1291.668: [DefNew: 48240K->4514K(52480K), 0.0205993 secs] > 123686K->79959K(518528K), 0.0206722 secs] > 1313.524: [GC 1313.524: [DefNew: 51169K->254K(52480K), 0.0317320 secs] > 126615K->78344K(518528K), 0.0317929 secs] > 1344.525: [GC 1344.525: [DefNew: 46910K->1729K(52480K), 0.0149128 secs] > 125000K->79820K(518528K), 0.0149729 secs] > 1351.479: [GC 1351.479: [DefNew: 48381K->4582K(52480K), 0.0218740 secs] > 126471K->82673K(518528K), 0.0219304 secs] > 1356.378: [Full GC 1356.379: [Tenured: 78090K->68675K(466048K), > 6.8007345secs] 100262K->68675K(518528K), [Perm : > 24575K->24575K(24576K)], > 6.8080206 secs] > 1368.802: [GC 1368.802: [DefNew: 46653K->4549K(52480K), 0.0286182 secs] > 115328K->73225K(518528K), 0.0286805 secs] > 1374.837: [GC 1374.837: [DefNew: 51205K->5688K(52480K), 0.0866753 secs] > 119881K->76341K(518528K), 0.0867317 secs] > 1380.416: [GC 1380.416: [DefNew: 52344K->5824K(52480K), 0.1281856 secs] > 122997K->78947K(518528K), 0.1282481 secs] > 1387.302: [GC 1387.302: [DefNew: 52479K->5824K(52480K), 0.1806688 secs] > 125603K->81752K(518528K), 0.1807336 secs] > 1393.171: [GC 1393.171: [DefNew: 52479K->5823K(52480K), 0.0392198 secs] > 128408K->84507K(518528K), 0.0392715 secs] > 1400.469: [GC 1400.469: [DefNew: 52479K->4437K(52480K), 0.0609195 secs] > 131163K->85732K(518528K), 0.0609793 secs] > 1403.957: [Full GC 1403.958: [Tenured: 81295K->76875K(466048K), > 0.3270033secs] 106066K->76875K(518528K), [Perm : > 28671K->28671K(28672K)], > 0.3270935 secs] > 1410.753: [GC 1410.753: [DefNew: 46656K->4605K(52480K), 0.0423694 secs] > 123531K->81480K(518528K), 0.0424263 secs] > 1416.358: [GC 1416.358: [DefNew: 51257K->5458K(52480K), 0.0375570 secs] > 128133K->86701K(518528K), 0.0376129 secs] > 1421.792: [GC 1421.792: [DefNew: 52114K->5339K(52480K), 0.1025094 secs] > 133357K->91866K(518528K), 0.1025714 secs] > 1538.764: [Full GC 1538.764: [Tenured: 86527K->59881K(466048K), > 5.3928429secs] 133369K->59881K(518528K), [Perm : > 32768K->32610K(32768K)], > 5.3929301 secs] > 1551.420: [GC 1551.420: [DefNew: 46655K->1848K(52480K), 0.0192879 secs] > 106537K->61729K(518528K), 0.0193421 secs] > 1557.712: [Full GC 1557.712: [Tenured: 59881K->64638K(466048K), > 0.3245996secs] 99584K->64638K(518528K), [Perm : > 36863K->36863K(36864K)], > 0.3246770 secs] > 1599.653: [GC 1599.653: [DefNew: 46655K->2500K(52480K), 0.0269780 secs] > 111294K->67139K(518528K), 0.0270333 secs] > 1603.423: [Full GC 1603.423: [Tenured: 64638K->67911K(466048K), > 0.3214652secs] 77743K->67911K(518528K), [Perm : > 40959K->40959K(40960K)], > 0.3215498 secs] > 1627.139: [GC 1627.139: [DefNew: 46655K->1289K(52480K), 0.0142188 secs] > 114567K->69201K(518528K), 0.0142742 secs] > 1670.137: [Full GC 1670.137: [Tenured: 67911K->69866K(466048K), > 0.3690838secs] 90651K->69866K(518528K), [Perm : > 45054K->45054K(45056K)], > 0.3691751 secs] > 1866.589: [GC 1866.589: [DefNew: 46418K->1563K(52480K), 0.0130402 secs] > 116284K->71430K(518528K), 0.0131005 secs] > 2324.705: [GC 2324.705: [DefNew: 48219K->4327K(52480K), 0.1842139 secs] > 118086K->74194K(518528K), 0.1842687 secs] > 2329.115: [GC 2329.115: [DefNew: 50983K->3916K(52480K), 0.2362021 secs] > 120850K->75941K(518528K), 0.2362599 secs] > 2332.533: [GC 2332.533: [DefNew: 50572K->1846K(52480K), 2.1846159 secs] > 122597K->77692K(518528K), 2.1846729 secs] > 2338.547: [GC 2338.547: [DefNew: 48502K->5824K(52480K), 0.4543022 secs] > 124348K->84948K(518528K), 0.4543563 secs] > 2341.175: [GC 2341.175: [DefNew: 52480K->1973K(52480K), 0.0189354 secs] > 131604K->83257K(518528K), 0.0189910 secs] > . > . > . > . > . > 2346.742: [GC 2346.742: [DefNew: 52479K->463K(52480K), 0.0085631 secs] > 135685K->85193K(518528K), 0.0086173 secs] > 2348.384: [GC 2348.384: [DefNew: 47119K->929K(52480K), 0.0053222 secs] > 131849K->85658K(518528K), 0.0053758 secs] > 2350.048: [GC 2350.048: [DefNew: 47585K->2118K(52480K), 0.0071565 secs] > 132314K->86847K(518528K), 0.0072113 secs] > 2351.522: [GC 2351.522: [DefNew: 48774K->1679K(52480K), 0.0072613 secs] > 133503K->86409K(518528K), 0.0073149 secs] > 2353.277: [GC 2353.277: [DefNew: 48335K->5823K(52480K), 0.1749974 secs] > 133065K->94648K(518528K), 0.1750549 secs] > 2355.637: [GC 2355.637: [DefNew: 52479K->3109K(52480K), 0.0142968 secs] > 141304K->92617K(518528K), 0.0143513 secs] > 2357.129: [Full GC 2357.129: [Tenured: 89508K->71494K(466048K), > 7.1956506secs] 105965K->71494K(518528K), [Perm : > 49151K->48973K(49152K)], > 7.2276270 secs] > 2369.631: [GC 2369.631: [DefNew: 46655K->1199K(52480K), 0.0261916 secs] > 118150K->72693K(518528K), 0.0262450 secs] > 2452.783: [GC 2452.783: [DefNew: 47855K->2579K(52480K), 0.0135995 secs] > 119349K->74073K(518528K), 0.0136626 secs] > 2457.584: [GC 2457.584: [DefNew: 49235K->5823K(52480K), 0.0369991 secs] > 120729K->78211K(518528K), 0.0370544 secs] > 2461.478: [GC 2461.478: [DefNew: 52479K->1589K(52480K), 0.0268260 secs] > 124867K->79759K(518528K), 0.0268830 secs] > 2464.523: [GC 2464.523: [DefNew: 48245K->1914K(52480K), 0.0117291 secs] > 126415K->80083K(518528K), 0.0117836 secs] > 2470.175: [GC 2470.175: [DefNew: 48570K->3163K(52480K), 0.0165518 secs] > 126739K->81333K(518528K), 0.0166066 secs] > 2473.213: [GC 2473.213: [DefNew: 49819K->2853K(52480K), 0.0148212 secs] > 127989K->82160K(518528K), 0.0148874 secs] > 2474.596: [GC 2474.596: [DefNew: 49509K->3104K(52480K), 0.0218307 secs] > 128816K->82411K(518528K), 0.0218863 secs] > 2478.481: [GC 2478.481: [DefNew: 49760K->3208K(52480K), 0.0126334 secs] > 129067K->82833K(518528K), 0.0126879 secs] > 2479.982: [GC 2479.983: [DefNew: 49864K->5824K(52480K), 0.1148272 secs] > 129489K->92140K(518528K), 0.1148850 secs] > 2481.667: [GC 2481.667: [DefNew: 52479K->3775K(52480K), 0.0376813 secs] > 138796K->91694K(518528K), 0.0377389 secs] > 2483.403: [GC 2483.403: [DefNew: 50431K->5823K(52480K), 0.1317053 secs] > 138350K->96769K(518528K), 0.1317564 secs] > 2485.690: [Full GC 2485.690: [Tenured: 90945K->91468K(466048K), > 3.2931426secs] 138286K->91468K(518528K), [Perm : > 53247K->53247K(53248K)], > 3.2932258 secs] > 2490.649: [GC 2490.649: [DefNew: 46655K->5824K(52480K), 0.0365206 secs] > 138124K->99492K(518528K), 0.0365753 secs]. > . > . > . > . > . > > 2888.080: [GC 2888.080: [DefNew: 47683K->2102K(52480K), 0.0122331 secs] > 168795K->123214K(518528K), 0.0122898 secs] > 2889.591: [GC 2889.591: [DefNew: 48758K->2383K(52480K), 0.0198578 secs] > 169870K->123496K(518528K), 0.0199112 secs] > 2891.947: [GC 2891.947: [DefNew: 49039K->3107K(52480K), 0.0112011 secs] > 170152K->124219K(518528K), 0.0112553 secs] > 2893.455: [GC 2893.455: [DefNew: 49763K->5718K(52480K), 0.0198187 secs] > 170875K->127604K(518528K), 0.0198690 secs] > 2895.015: [GC 2895.015: [DefNew: 52374K->5824K(52480K), 0.0642596 secs] > 174260K->132189K(518528K), 0.0643112 secs] > 2896.954: [GC 2896.954: [DefNew: 52480K->3493K(52480K), 0.0117051 secs] > 178845K->130019K(518528K), 0.0117596 secs] > 2899.660: [GC 2899.660: [DefNew: 50149K->1101K(52480K), 0.0075071 secs] > 176675K->127790K(518528K), 0.0075585 secs] > 2901.454: [GC 2901.454: [DefNew: 47756K->587K(52480K), 0.0068498 secs] > 174445K->127276K(518528K), 0.0069012 secs] > 2905.571: [Full GC 2905.571: [Tenured: 126688K->100740K(466048K), > 10.5908514secs] 157839K->100740K(518528K), [Perm : > 65535K->65063K(65536K)], > 10.6232907 secs] > 2919.226: [Full GC 2919.226: [Tenured: 100740K->101030K(466048K), > 0.5165606secs] 138033K->101030K(518528K), [Perm : > 65535K->65535K(65536K)], > 0.5166402 secs] > 2919.748: [Full GC 2919.748: [Tenured: 101030K->101030K(466048K), > 0.5793060secs] 101030K->101030K(518528K), [Perm : > 65535K->65535K(65536K)], > 0.5794041 secs] > 2920.335: [Full GC 2920.335: [Tenured: 101030K->101030K(466048K), > 0.4588885secs] 101030K->101030K(518528K), [Perm : > 65535K->65535K(65536K)], > 0.4589650 secs] > 2920.802: [Full GC 2920.802: [Tenured: 101030K->97189K(466048K), > 0.6423210secs] 101030K->97189K(518528K), [Perm : > 65535K->65497K(65536K)], > 0.6424079 secs] > 2968.834: [Full GC 2968.834: [Tenured: 97189K->97335K(466048K), > 0.6619274secs] 99370K->97335K(518528K), [Perm : > 65535K->65535K(65536K)], > 0.6620179 secs] > 2969.556: [Full GC 2969.556: [Tenured: 97335K->97396K(466048K), > 0.4763756secs] 97664K->97396K(518528K), [Perm : > 65535K->65535K(65536K)], > 0.4764480 secs] > > > when i remove some 30-odd *.xml deployment files from /conf.../localhost and > have it load just TWO webapps, it lasts much longer > > what should i do? is this a PermGen problemo? And if it is, why does my > error message keep saying "ServeletException" and not the OutOfMemory > PermGen message? > > thanks - and if anyone needs graphs/files/blood i'll be happy to drop them > somewhere > > AC > > On 10/10/07, Pid <[EMAIL PROTECTED]> wrote: > > > > Christopher Schultz wrote: > > > A C, > > > > > > A C wrote: > > >> We've been experiencing horrendous OOMEs recently after deploying 40 > > clones > > >> of a vendor's .war file which encompasses their > > application/portal. The > > >> context is about 100MB > > > > > > Lesee... 100 MB times 40 instances .. mm... carry the 7 ... hey, that's > > > 4GB! Where have I heard that number before... hmm.. something about .. > > > right! The maximum process memory on a 32-bit OS! > > > > lol > > > > > > > > A C, I have to say this: the clue is in the question... > > > > I'll summarise; it works OK when when you deploy one copy of a 100Mb > > app, but when you deploy FORTY it gets a bit messy and you run out of > > memory. > > > > How is it that you're not thinking about more servers at this point? > > > > (I'd be waving a nice diagram at the accountants and saying 'shiny new > > boxes please') > > > > > > p > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To start a new topic, e-mail: users@tomcat.apache.org > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > --------------------------------------------------------------------- > > To start a new topic, e-mail: users@tomcat.apache.org > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
