We've been using a modified DataPort w/Jetty for a while and never had any problems.
However, today we tried running it under Tomcat and it failed. Paraphrased from my source, I do the following: SelectQuery select = new SelectQuery(entity); select.setFetchingDataRows(true); IteratedSelectObserver observer = new IteratedSelectObserver(); sourceNode.performQueries(Collections.singletonList(select), observer); ResultIterator result = observer.getResultIterator(); if (result.hasNextRow()) { // works fine w/jetty, but throws exception below w/Tomcat 5.5.20 Map nextRow = result.nextDataRow(); The DataSource is configured in Tomcat as follows: <Resource name="jdbc/qpbuser" auth="Container" type="javax.sql.DataSource" driverClassName="org.apache.derby.jdbc.EmbeddedDriver" url="jdbc:derby:c:\workspace\qpbtapestry\testdb\demo_derby;create=true" username="" password="" maxActive="5" maxIdle="2"/> In the Tomcat log I find the below: 56875 [http-8080-Processor25] INFO org.objectstyle.cayenne.access.QueryLogger - --- will run 1 query. 56875 [http-8080-Processor25] INFO org.objectstyle.cayenne.access.QueryLogger - SELECT t0.TREATASPECR_ID, t0.TREATASPEC_ACTIVE, t0.TREATASPEC_CRITID, t0.TREATASPEC_FIXEDCOST, t0.TREATASPEC_ID, t0.TREATASPEC_QTYPE, t0.TREATASPEC_VC1, t0.TREATASPEC_VC2, t0.TREATASPEC_VC2CRITID, t0.TREATASPEC_VC2FIXEDCOST, t0.TREATASPEC_VC2QTYPE, t0.TREATASPEC_VC3, t0.TREATASPEC_VC3CRITID, t0.TREATASPEC_VC3FIXEDCOST, t0.TREATASPEC_VC3QTYPE, t0.TREATASPEC_VC4, t0.TREATASPEC_VC4CRITID, t0.TREATASPEC_VC4FIXEDCOST, t0.TREATASPEC_VC4QTYPE, t0.TREATASPEC_VC5, t0.treataspec_rolehastoclose FROM TREAT_AREASPEC t0 org.objectstyle.cayenne.CayenneException: [v.1.2.1 August 30 2006] Exception materializing column. at org.objectstyle.cayenne.access.jdbc.JDBCResultIterator.readDataRow(JDBCResultIterator.java:342) at org.objectstyle.cayenne.access.jdbc.JDBCResultIterator.nextDataRow(JDBCResultIterator.java:181) at com.zylin.payback.app.db.data.tools.DataPort.processInsert(DataPort.java:204) at com.zylin.payback.app.db.data.tools.DataPort.execute(DataPort.java:106) at com.zylin.payback.app.html.Backup.execute(Backup.java:162) at com.zylin.payback.app.html.Backup.backup(Backup.java:73) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.tapestry.listener.ListenerMap.invokeTargetMethod(ListenerMap.java:257) at org.apache.tapestry.listener.ListenerMap.access$100(ListenerMap.java:46) at org.apache.tapestry.listener.ListenerMap$SyntheticListener.invoke(ListenerMap.java:97) at org.apache.tapestry.listener.ListenerMap$SyntheticListener.actionTriggered(ListenerMap.java:102) at org.apache.tapestry.link.DirectLink.trigger(DirectLink.java:119) at org.apache.tapestry.engine.DirectService.service(DirectService.java:169) at org.apache.tapestry.engine.AbstractEngine.service(AbstractEngine.java:889) at org.apache.tapestry.ApplicationServlet.doService(ApplicationServlet.java:198) at org.apache.tapestry.ApplicationServlet.doGet(ApplicationServlet.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at com.zylin.payback.app.tapestry.SequentialSessionFilter.doFilter(SequentialSessionFilter.java:27) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Unknown Source) Caused by: SQL Exception: ResultSet not open. Operation 'getXXX' not permitted. Verify that autocommit is OFF. at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.newSQLException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.checkIfClosed(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.getColumn(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.getString(Unknown Source) at org.apache.tomcat.dbcp.dbcp.DelegatingResultSet.getString(DelegatingResultSet.java:174) at org.objectstyle.cayenne.access.types.CharType.materializeObject(CharType.java:152) at org.objectstyle.cayenne.access.jdbc.JDBCResultIterator.readDataRow(JDBCResultIterator.java:327) ... 37 more This is the interesting part of our modified DataPort routine: /** * Reads source data from source, saving it to destination. */ protected void processInsert(List entities) throws CayenneException { // Allow delegate to modify the list of entities // any way it wants. For instance delegate may filter // or sort the list (though it doesn't have to, and can simply // pass through the original list). if (entities == null || entities.isEmpty()) { return; } // process ordered list of entities one by one Iterator it = entities.iterator(); while (it.hasNext()) { DbEntity entity = (DbEntity) it.next(); // skip derived DbEntities... if (entity instanceof DerivedDbEntity) { continue; } try { SelectQuery select = new SelectQuery(entity); select.setFetchingDataRows(true); // Using QueryResult as observer for the data insert. // This allows to collect query statistics and pass it to the delegate. QueryResult insertObserver = new QueryResult(); insertObserver.clear(); // Create an observer for to get the iterated result // instead of getting each table as a list // delegate is allowed to substitute query IteratedSelectObserver observer = new IteratedSelectObserver(); sourceNode.performQueries(Collections.singletonList(select), observer); ResultIterator result = observer.getResultIterator(); try { // Batch size 1. while (result.hasNextRow()) { // end of the batch detected... commit and start a new // insert query InsertBatchQuery insert = new InsertBatchQuery(entity, 1); insertObserver.clear(); Map nextRow = result.nextDataRow(); insert.add(nextRow); try { destinationNode.performQueries(Collections .singletonList(insert), insertObserver); } catch (Exception e) { e.printStackTrace(); } } } finally { try { // don't forget to close ResultIterator result.close(); } catch (CayenneException ex) { } } } catch (CayenneException e) { /* Continue anyway */ e.printStackTrace(); } } } -- Øyvind Harboe http://www.zylin.com