Re: [PERFORM] Slow count(*) again...

2010-10-14 Thread Neil Whelchel
On Wednesday 13 October 2010 06:27:34 you wrote: > On Wed, Oct 13, 2010 at 4:38 AM, Neil Whelchel wrote: > > There seems to be allot of discussion about VACUUM FULL, and its > > problems. The overall buzz seems to be that VACUUM FULL is a bad idea (I > > could be wrong he

Re: [PERFORM] Slow count(*) again...

2010-10-13 Thread Neil Whelchel
On Wednesday 13 October 2010 05:33:28 Mladen Gogala wrote: > On 10/13/2010 8:12 AM, Greg Smith wrote: > > The work incorporating a more stable XFS into RHEL started with xfsprogs > > 3.0.1-6 going into Fedora 11, and 3.1.X would represent a current > > release. So your Ubuntu kernel is two major

Re: [PERFORM] Slow count(*) again...

2010-10-13 Thread Neil Whelchel
On Wednesday 13 October 2010 01:50:23 Mark Kirkwood wrote: > On 13/10/10 21:38, Neil Whelchel wrote: > > So with our conclusion pile so far we can deduce that if we were to keep > > all of our data in two column tables (one to link them together, and the > > other to store on

Re: [PERFORM] Slow count(*) again...

2010-10-13 Thread Neil Whelchel
On Wednesday 13 October 2010 00:19:26 Mark Kirkwood wrote: > On 13/10/10 19:47, Neil Whelchel wrote: > > Nope... > > So, possible conclusions are: > > 1. Even with VACUUM database table speed degrades as tables are updated. > > 2. Time testing on a freshly INSERTed table

Re: [PERFORM] Slow count(*) again...

2010-10-12 Thread Neil Whelchel
On Sunday 10 October 2010 21:15:56 Neil Whelchel wrote: > Right now, I am building a test machine with two dual core Intel processors > and two 15KRPM mirrored hard drives, 1 GB ram. I am using a small amount of > ram because I will be using small test tables. I may do testing in the

Re: [PERFORM] Slow count(*) again...

2010-10-12 Thread Neil Whelchel
On Tuesday 12 October 2010 14:35:01 you wrote: > > suggest that 99% instances of the "select count(*)" idiom are probably > > > >> bad use of the SQL language. > > Well, suppose you paginate results. If the user sees that the search query > returns 500 pages, there are two options : > > - you're

Re: [PERFORM] Slow count(*) again...

2010-10-12 Thread Neil Whelchel
On Tuesday 12 October 2010 08:39:19 Dan Harris wrote: > On 10/11/10 8:02 PM, Scott Carey wrote: > > would give you a 1MB read-ahead. Also, consider XFS and its built-in > > defragmentation. I have found that a longer lived postgres DB will get > > extreme file fragmentation over time and sequen

Re: [PERFORM] Slow count(*) again...

2010-10-12 Thread Neil Whelchel
On Tuesday 12 October 2010 07:19:57 you wrote: > >> The biggest single problem with "select count(*)" is that it is > >> seriously overused. People use that idiom to establish existence, which > >> usually leads to a performance disaster in the application using it, > >> unless the table has no mor

Re: [PERFORM] Slow count(*) again...

2010-10-11 Thread Neil Whelchel
On Monday 11 October 2010 19:23:46 Mladen Gogala wrote: > On 10/11/2010 10:02 PM, Scott Carey wrote: > > Did you tune the linux FS read-ahead first? You can get large gains by > > doing that if you are on ext3. blockdev --setra 2048 > > Actually, I have blockdev --setra 32768 > > > would give

Re: [PERFORM] Slow count(*) again...

2010-10-11 Thread Neil Whelchel
On Monday 11 October 2010 16:50:36 you wrote: > On 10/11/2010 3:54 PM, Neil Whelchel wrote: > > 1. A faster count(*), or something like my proposed estimate(*). > > 2. A way to get the total rows matched when using LIMIT and OFFSET before > > LIMIT and OFFSET are applied. &g

Re: [PERFORM] Slow count(*) again...

2010-10-11 Thread Neil Whelchel
On Monday 11 October 2010 10:46:17 Craig James wrote: > On 10/9/10 6:47 PM, Scott Marlowe wrote: > > On Sat, Oct 9, 2010 at 5:26 PM, Neil Whelchel wrote: > >> I know that there haven been many discussions on the slowness of > >> count(*) even when an index is involve

Re: [PERFORM] Slow count(*) again...

2010-10-10 Thread Neil Whelchel
On Sunday 10 October 2010 15:41:16 you wrote: > On 10/11/2010 01:14 AM, Mladen Gogala wrote: > > I can provide measurements, but from Oracle RDBMS. Postgres doesn't > > allow tuning of that aspect, so no measurement can be done. Would the > > numbers from Oracle RDBMS be acceptable? > > Well, they

Re: [PERFORM] Slow count(*) again...

2010-10-10 Thread Neil Whelchel
On Sunday 10 October 2010 05:02:03 Віталій Тимчишин wrote: > 2010/10/10 Neil Whelchel > > > On Saturday 09 October 2010 18:47:34 Scott Marlowe wrote: > > > On Sat, Oct 9, 2010 at 5:26 PM, Neil Whelchel > > > > wrote: > > > > I know that there h

Re: [PERFORM] Slow count(*) again...

2010-10-10 Thread Neil Whelchel
On Saturday 09 October 2010 23:56:15 Craig Ringer wrote: > On 10/10/2010 11:02 AM, Neil Whelchel wrote: > > On Saturday 09 October 2010 18:47:34 Scott Marlowe wrote: > >> On Sat, Oct 9, 2010 at 5:26 PM, Neil Whelchel > > > > wrote: > >>> I know tha

Re: [PERFORM] Slow count(*) again...

2010-10-09 Thread Neil Whelchel
On Saturday 09 October 2010 18:47:34 Scott Marlowe wrote: > On Sat, Oct 9, 2010 at 5:26 PM, Neil Whelchel wrote: > > I know that there haven been many discussions on the slowness of count(*) > > even when an index is involved because the visibility of the rows has to > > be

[PERFORM] Slow count(*) again...

2010-10-09 Thread Neil Whelchel
I know that there haven been many discussions on the slowness of count(*) even when an index is involved because the visibility of the rows has to be checked. In the past I have seen many suggestions about using triggers and tables to keep track of counts and while this works fine in a situation