Hello, Thank you for everyone's input.
I had originally taken out the sub queries from the main method and moved them to another method(IE check if city is there if not add it and link the foreign key) and then moved it to a map held in memory(still putting unknown ones in the db) so there were less calls to the db. This didn't help but then I moved back to the @CommitAfter(which was the original way I had started) and it worked fine. I like your clear idea and creating my own transactions. I think I'll give that a go as well. --James -----Original Message----- From: Jonathan Barker [mailto:jonathan.theit...@gmail.com] Sent: January-16-09 12:31 AM To: 'Tapestry users' Subject: RE: T5 and hibernate slowing with more records Been there, hit that. You need to clear() the session. Or, you can also evict() as you go. Basically, every time Hibernate needs to commit something, it needs to flush to the database, and it looks at all of the Hibernate objects that it knows about - not just those to which you have current references. For every object you handle, you add one more that it needs to check to see if it needs to have changes flushed. The clear() gets rid of anything you are not holding a reference to. I tend to do it every ten records or so, but I've never tried to optimize it. The other option, as someone suggested, is to commit a batch rather than single records, but that can be nasty for large sets. I'm working with an import of about 250,000 records as I write this. Jonathan > -----Original Message----- > From: James Sherwood [mailto:jsherw...@rgisolutions.com] > Sent: Thursday, January 15, 2009 13:16 > To: 'Tapestry users' > Subject: T5 and hibernate slowing with more records > > Hello, > > > > I am using POI to take an excel file and put it into a Mysql database > using > hibernate. > > > > I put a counter on the records going into the database and it starts off > fast but then slows to over 15 seconds a record and I cannot figure out > why. > > > > My code is basically this: > > > > Public class DatabaseDump{ > > > > @Inject > > Private Session _session; > > > > Public void onAction(){ //they hit the go button > > Loop rows{ > > > _session.beginTransaction(); > > Loop cells{ > > Fill the > object from the cells > > > _session.save(Object); > > } > > > _session.getTransaction().commit(); > > } > > } > > > > Everything works fine as in the database gets filled for the rows it has > completed but gradually slows to a crawl and at around 2700 records of > 10500 > records it's a record every 15 seconds. > > > > I run the project under eclipse using Tomcat. The memory starts at around > 80mb and only goes up to around 110mb. > > > > What am I doing wrong? > > > > --James > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
