Re: paged query slow when fetching big lists

2007-07-05 Thread Andrus Adamchik
3.0 (SVN trunk), which will hopefully be released fairly soon as 3.0M1 Andrus On Jul 5, 2007, at 3:06 PM, Borut Bolčina wrote: Hello, which version includes this corrections? 2.0.3 or 3.0? Thanks, Borut On 26.6.2007 20:32, Andrus Adamchik wrote: Hi Marcin, I have good news (I think). Fetc

Re: paged query slow when fetching big lists

2007-07-05 Thread Borut Bolčina
Hello, which version includes this corrections? 2.0.3 or 3.0? Thanks, Borut On 26.6.2007 20:32, Andrus Adamchik wrote: Hi Marcin, I have good news (I think). Fetching just id columns inside the IncrementalFaultList indeed speeds things up significantly. I just committed the change to o

Re: paged query slow when fetching big lists

2007-06-29 Thread Michael Gentry
Not to pick on your iMac, which is a nice machine, but the PowerPC line was notoriously bad at running Java (G5 better than the G4, obviously). If you tried this on an Intel box, I suspect your speed improvement would be even better (I'm guessing a good portion of your 10 seconds is in the Java G

Re: paged query slow when fetching big lists

2007-06-28 Thread Aristedes Maniatis
On 28/06/2007, at 6:10 PM, Andrus Adamchik wrote: Fixed. Now the things are fast on the client as well. A final followup on these changes. We've done testing both single user and three-tier and the speed improvement is now consistent throughout. With the following setup: * 1.6Ghz iMac

Re: AW: paged query slow when fetching big lists

2007-06-28 Thread Andrus Adamchik
such a support! -Ursprüngliche Nachricht- Von: Andrus Adamchik [mailto:[EMAIL PROTECTED] Gesendet: Donnerstag, 28. Juni 2007 10:11 An: user@cayenne.apache.org Betreff: Re: paged query slow when fetching big lists Fixed. Now the things are fast on the client as well. Andrus On Jun 28, 2007

AW: paged query slow when fetching big lists

2007-06-28 Thread Peter Schröder
the speed of fixing things is amaizing... i wish that tapestry would have such a support! -Ursprüngliche Nachricht- Von: Andrus Adamchik [mailto:[EMAIL PROTECTED] Gesendet: Donnerstag, 28. Juni 2007 10:11 An: user@cayenne.apache.org Betreff: Re: paged query slow when fetching big lists

Re: paged query slow when fetching big lists

2007-06-28 Thread Andrus Adamchik
Fixed. Now the things are fast on the client as well. Andrus On Jun 28, 2007, at 12:18 AM, Andrus Adamchik wrote: On Jun 27, 2007, at 6:38 AM, Marcin Skladaniec wrote: There is one but: fix does work only for queries executed on server, when I executed the query on (ROP) client, the query

Re: paged query slow when fetching big lists

2007-06-27 Thread Andrus Adamchik
On Jun 27, 2007, at 6:38 AM, Marcin Skladaniec wrote: There is one but: fix does work only for queries executed on server, when I executed the query on (ROP) client, the query takes the same amount of time ! Is it possible that the remote calls are using a different constructor ? or maybe

Re: paged query slow when fetching big lists

2007-06-26 Thread Marcin Skladaniec
Hi Andrus ! The fix you committed today works great, I can confirm, in our case the 30-50 second SelectQuery now executes in 5-7 seconds ! There is one but: fix does work only for queries executed on server, when I executed the query on (ROP) client, the query takes the same amount of tim

Re: paged query slow when fetching big lists

2007-06-26 Thread Andrus Adamchik
Hi Marcin, I have good news (I think). Fetching just id columns inside the IncrementalFaultList indeed speeds things up significantly. I just committed the change to optimize SelectQueries to do just that. Please let me know how does it work for you. Now the profiling details... * I prof

Re: paged query slow when fetching big lists

2007-06-26 Thread Marcin Skladaniec
Hi Andrus ! Many thanks for that ! Marcin On 26/06/2007, at 5:39 PM, Andrus Adamchik wrote: Hi Marcin, 1) SelectQuery(Student.class) with page size 10 takes 30-50 seconds. 2) SQLTemplate(Student.class, "SELECT #result('id' 'int') from STUDENT") without paging takes 100 sec. 3) SQLTemplate(S

Re: paged query slow when fetching big lists

2007-06-26 Thread Andrus Adamchik
Hi Marcin, 1) SelectQuery(Student.class) with page size 10 takes 30-50 seconds. 2) SQLTemplate(Student.class, "SELECT #result('id' 'int') from STUDENT") without paging takes 100 sec. 3) SQLTemplate(Student.class, "SELECT #result('id' 'int') from STUDENT") with page size 10 takes 5 sec. 4) SQ

Re: paged query slow when fetching big lists

2007-06-25 Thread Marcin Skladaniec
Hi I have done some more profiling and testing. executing queries on table with >10 records, directly on server (not on client) gave results as listed below: 1) SelectQuery(Student.class) with page size 10 takes 30-50 seconds. 2) SQLTemplate(Student.class, "SELECT #result('id' 'int') fro

Re: paged query slow when fetching big lists

2007-06-25 Thread Marcin Skladaniec
Hi Andrus I had not much time to check, but with the fix the 100k records load in 30 instead of 50 seconds. It is some improvement, but not enough. I'll do some more profiling tomorrow and let you know. By the way, we are using netbeans for profiling, the benefit : it is free. I will eval

Re: paged query slow when fetching big lists

2007-06-23 Thread Andrus Adamchik
You need to use a profiler. You can try an evaluation version of YourKit for example. It has a nice Eclipse plugin, so you can profile straight from Eclipse: http://www.yourkit.com/ (wonder if ASF has a free open source license for YourKit? ObjectStyle does - it was granted to Cayenne

Re: paged query slow when fetching big lists

2007-06-23 Thread Andrus Adamchik
Ari, Marcin -- going through the code I noticed one inefficiency - the elements array access is synchronized in 'fillIn' method. Since 'fillIn' is called from constructor, such synchronization is unneeded and only slows things down. I just checked a fixed version to trunk. Could you try i

Re: paged query slow when fetching big lists

2007-06-22 Thread Aristedes Maniatis
On 22/06/2007, at 11:10 PM, Michael Gentry wrote: Marcin, this thread might be of interest to you ... http://mail-archives.apache.org/mod_mbox/cayenne-dev/200705.mbox/ browser Look at the "Paging and SQL queries" thread on May 25. Yes, this is the same project we are working on. I starte

Re: paged query slow when fetching big lists

2007-06-22 Thread Andrus Adamchik
Hi Marcin, * fetch only Pk columns and create all ObjectIds at once, get rid of the iterating process if possible * use already existing method resolveInterval() to fault the required range of records This strategy was discussed in the May thread with Ari (the one that Michael Gentry ment

Re: paged query slow when fetching big lists

2007-06-22 Thread Michael Gentry
Marcin, this thread might be of interest to you ... http://mail-archives.apache.org/mod_mbox/cayenne-dev/200705.mbox/browser Look at the "Paging and SQL queries" thread on May 25. /dev/mrg

paged query slow when fetching big lists

2007-06-21 Thread Marcin Skladaniec
Hi Recently we have found that fetching a list of 100,000 records using ROP with paging and no cache takes a long time, about 50 seconds in our case. We have profiled the cpu usage and the result shows that 99% of time is spent in IncrementalFaultList, within the fillIn() method. The fil