On Apr 1, 2013, at 4:18 PM, David Landis wrote: > Thanks for the response, see my comments inline below. > > > On Mon, Apr 1, 2013 at 3:49 PM, Daniel Mikusa <dmik...@vmware.com> wrote: > >> On Apr 1, 2013, at 3:31 PM, David Landis wrote: >> >>> Hi guys, >>> >>> When running a performance test on my system it starts fine, but after a >>> while I start getting errors in my application log such as (see the >> bottom >>> for full stack trace): >>> >>> 2013-03-29 16:38:54,778 ERROR >>> [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] - >>> [SimpleAsyncTaskExecutor-12842] - [SimpleAsyncTaskExecutor-12842] >> Timeout: >>> Pool empty. Unable to fetch a connection in 30 seconds, none >>> available[size:80; busy:0; idle:0; lastwait:30000]. >> >> This means you have no connections in your pool and it's unable to create >> a new connection to your database. >> > > > OK, I'll have to investigate the DB setting more thoroughly. The maximum > sessions and processes in Oracle are higher than we were using for the test > though (several hundred). > > > >> >>> >>> Questions: >>> >>> 1.) I'm a little confused about what it means if no connections are >>> available and yet none are "busy" nor "idle". What are the other >> available >>> states? >> >> The pool is empty. Further more the error above means that it can't >> create a new connection either. Maybe your network failed? or the DB >> kicked off all your application's connections? >> > > > Actually Oracle was showing 70+ inactive sessions for my app even though > the connection pool was showing empty.
Possible you are hitting a bug. You might also want to try an upgrade of Tomcat. You're a couple versions back at 7.0.32. You can see what was fixed by searching for "jdbc-pool" in the ChangeLog. https://tomcat.apache.org/tomcat-7.0-doc/changelog.html Dan > > >> Were you ever able to get a connection to the DB? If you restart Tomcat, >> can you get connections to the DB again? >> > > > Yes, restarting Tomcat results in a fresh pool of DB connections and the > 70+ inactive sessions on the DB side are gone and replaced by 10 which is > the initial size of the pool. > > > >> >> Also, are there any limits on your DB user's account that might cause >> problems with your performance tests? >> > > > Not that I know of, but I'll look further. I was expecting problems with > the perf test eventually b/c it was set to simulate a couple hundred users > and I only maxActive set to 80. That is fine. I'm more concerned with why > the connection pool didn't eventually recover. > > > >> >>> >>> 2.) My other point of confusion is that assuming there is a connection >> leak >>> in the application, shouldn't setting removeAbandoned=true cause the DB >>> connections to eventually be recovered? >> >> Yes. >> >>> What I am seeing is that even after >>> a couple days of no application usage now I'm still getting that exact >>> error seen above when trying to use the application. >> >> See above, something bad is happening in your environment, worse than your >> application holding onto a connection. You're going to need to >> troubleshoot further and see why you can't make connections to the DB. >> >> Dan >> >> >> >>> >>> Thanks in advance. >>> >>> ---------------------------------------- >>> >>> environment: >>> Tomcat 7.0.32 >>> RedHat Linux 64 bit >>> Java 7 >>> >>> context.xml: >>> >>> <Resource name="fdatasource" >>> auth="Container" >>> type="javax.sql.DataSource" >>> factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" >>> testWhileIdle="true" >>> testOnBorrow="true" >>> testOnReturn="true" >>> validationQuery="select 1 from dual" >>> validationInterval="30000" >>> timeBetweenEvictionRunsMillis="5000" >>> maxActive="80" >>> minIdle="10" >>> maxWait="30000" >>> initialSize="10" >>> removeAbandonedTimeout="1200" >>> removeAbandoned="true" >>> logAbandoned="true" >>> minEvictableIdleTimeMillis="60000" >>> jmxEnabled="false" >>> jdbcInterceptors="StatementFinalizer" >>> username="${db.user}" >>> password="${db.pw}" >>> driverClassName="oracle.jdbc.OracleDriver" >>> url="${db.url}" /> >>> >>> example stacktrace: >>> >>> Mar 29, 2013 5:13:34 PM org.apache.catalina.core.StandardWrapperValve >> invoke >>> SEVERE: Servlet.service() for servlet [dispatcher] in context with path >>> [/app] threw exception [Request processing failed; nested exception is >>> org.springframework.transaction.CannotCreateTransactionException: Could >> not >>> open Hibernate Session for transaction; nested exception is >>> org.hibernate.exception.GenericJDBCException: Could not open connection] >>> with root cause >>> org.apache.tomcat.jdbc.pool.PoolExhaustedException: >>> [http-apr-8443-exec-158] Timeout: Pool empty. Unable to fetch a >> connection >>> in 30 seconds, none available[size:84; busy:0; idle:0; lastwait:30000]. >>> at >>> >> org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:675) >>> at >>> >> org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:188) >>> at >>> >> org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128) >>> at >>> >> org.hibernate.service.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:141) >>> at >>> >> org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292) >>> at >>> >> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:297) >>> at >>> >> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:169) >>> at >>> >> org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67) >>> at >>> >> org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:160) >>> at >>> >> org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1395) >>> at >>> >> org.springframework.orm.hibernate4.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:387) >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
