Re: [PERFORM] Scalability in postgres

2009-06-01 Thread Greg Smith
On Sat, 30 May 2009, Scott Marlowe wrote: 8.04 was a frakking train wreck in many ways. It wasn't until 8.04.2 came out that it was even close to useable as a server OS, and even then, not for databases yet. It's still got broken bits and pieces marked "fixed in 8.10"... Uh, hello, it's your

Re: [PERFORM] Scalability in postgres

2009-06-01 Thread Ron Mayer
Grzegorz Jaśkiewicz wrote: > > I thought that's where the difference is between postgresql and oracle > mostly, ability to handle more transactions and better scalability . > Which were you suggesting had this "better scalability"? I recall someone summarizing to a CFO where I used to work: "Or

Re: [PERFORM] Vacuuming technique doubt

2009-06-01 Thread Robert Haas
On Mon, Jun 1, 2009 at 8:35 PM, Kevin Grittner wrote: > S Arvind wrote: > >> The reason why we need it manually is , we don't need any >> performance drop in our production hours. So we figured out the most >> less usage working time, most freq used tables and want to perform >> that on daily . s

Re: [PERFORM] Vacuuming technique doubt

2009-06-01 Thread Kevin Grittner
S Arvind wrote: > The reason why we need it manually is , we don't need any > performance drop in our production hours. So we figured out the most > less usage working time, most freq used tables and want to perform > that on daily . so in weekends we can vaccum and reindex entire db.. By th

Re: [PERFORM] Unexpected query plan results

2009-06-01 Thread Robert Haas
On Mon, Jun 1, 2009 at 4:53 PM, Anne Rosset wrote: >> On Mon, Jun 1, 2009 at 2:14 PM, Anne Rosset wrote: >>> SELECT SUM(1) FROM item WHERE is_deleted = 'f'; sum - 1824592 (1 >>> row) >>> SELECT SUM(1) FROM item WHERE folder_id = 'tracker3641 >>> '; sum >>> 122412 (1 row) >>> SELE

Re: [PERFORM] Unexpected query plan results

2009-06-01 Thread Anne Rosset
Dave Dutcher wrote: -Original Message- From: Anne Rosset Subject: Re: [PERFORM] Unexpected query plan results SELECT SUM(1) FROM item WHERE is_deleted = 'f'; sum - 1824592 (1 row) SELECT SUM(1) FROM item WHERE folder_id = 'tracker3641 '; sum 122

Re: [PERFORM] Unexpected query plan results

2009-06-01 Thread Dave Dutcher
> -Original Message- > From: Anne Rosset > Subject: Re: [PERFORM] Unexpected query plan results > > >> > >>SELECT SUM(1) FROM item WHERE is_deleted = 'f'; sum > - 1824592 > >>(1 > >>row) > >>SELECT SUM(1) FROM item WHERE folder_id = 'tracker3641 > >>'; sum > >> > >>1224

Re: [PERFORM] Unexpected query plan results

2009-06-01 Thread Anne Rosset
Robert Haas wrote: On Mon, Jun 1, 2009 at 2:14 PM, Anne Rosset wrote: The table has 468173 rows and the value for default_statistics_target is 750. Anne Hi Robert, we did a vacuum analyze and the results are the same. Here are the results of the queries : SELECT SUM(1) FROM item

Re: [PERFORM] Unexpected query plan results

2009-06-01 Thread Robert Haas
On Mon, Jun 1, 2009 at 2:14 PM, Anne Rosset wrote: >>> The table has 468173 rows and the value for default_statistics_target is >>> 750. >>> Anne > Hi Robert, > we did a vacuum analyze and the results are the same. > Here are the results of the queries : > > SELECT SUM(1) FROM item WHERE is_delete

Re: [PERFORM] Scalability in postgres

2009-06-01 Thread Scott Carey
On 5/31/09 9:37 AM, "Fabrix" wrote: > > > 2009/5/29 Scott Carey >> >> On 5/28/09 6:54 PM, "Greg Smith" wrote: >> >>> 2) You have very new hardware and a very old kernel.  Once you've done the >>> above, if you're still not happy with performance, at that point you >>> should consider using

Re: [PERFORM] Unexpected query plan results

2009-06-01 Thread Anne Rosset
Robert Haas wrote: On Fri, May 29, 2009 at 5:57 PM, Anne Rosset wrote: Robert Haas wrote: On Thu, May 28, 2009 at 6:46 PM, Anne Rosset wrote: -> Index Scan using item_pk on item (cost=0.00..176865.31 rows=97498 width=88) (actu

Re: [PERFORM] Vacuuming technique doubt

2009-06-01 Thread S Arvind
Hi Smith, The reason why we need it manually is , we don't need any performance drop in our production hours. So we figured out the most less usage working time, most freq used tables and want to perform that on daily . so in weekends we can vaccum and reindex entire db.. Is the model is not more

Re: [PERFORM] Best way to load test a postgresql server

2009-06-01 Thread Alan McKay
Disclaimer : I'm very much a newbie here! But I am on the path in my new job to figure this stuff out as well, and went to PG Con here in Ottawa 2 weeks ago and attended quite a few lectures on this topic. Have a look at : http://wiki.postgresql.org/wiki/PgCon_2009 And in particular "Database H

Re: [PERFORM] Very inefficient query plan with disjunction in WHERE clause

2009-06-01 Thread Віталій Тимчишин
2009/6/1 Koen Martens > > Now, when I split up the OR in two distinct queries, everything is nice and > fast. Both queries run in sub-second time. Hi. PostgreSQL simply do not like ORs (can't use indexes in this case), so UNION/UNION ALL is your friend. Best regards, Vitalii Tymchyshyn

Re: [PERFORM] Using index for bitwise operations?

2009-06-01 Thread Tom Lane
Shaul Dar writes: > I have at column that is a bit array of 16, each bit specifying if a certain > property, out of 16, is present or not. Our typical query select 300 > "random" rows (could be located in different blocks) from the table based on > another column+index, and then filters them down

Re: [PERFORM] Using index for bitwise operations?

2009-06-01 Thread Richard Huxton
Shaul Dar wrote: Hi, I have at column that is a bit array of 16, each bit specifying if a certain property, out of 16, is present or not. Our typical query select 300 "random" rows (could be located in different blocks) from the table based on another column+index, and then filters them down to

[PERFORM] Very inefficient query plan with disjunction in WHERE clause

2009-06-01 Thread Koen Martens
Hi all, I've been staring at this for hours (if not days). Any hints are appreciated! At first I thought "there must be a way to make postgresql perform on this thing", but i've lost hope that pgsql actually can deal with it.. The query is: SELECT DISTINCT posrel.pos, questions.number, q

[PERFORM] Best way to load test a postgresql server

2009-06-01 Thread Peter Sheats
Hi, I¹m about to set up a large instance on Amazon EC2 to be our DB server. Before we switch to using it in production I would like to simulate some load on it so that I know what it can handle and so that I can make sure I have the optimal settings in the config file. What is the best strategy

[PERFORM] Using index for bitwise operations?

2009-06-01 Thread Shaul Dar
Hi, I have at column that is a bit array of 16, each bit specifying if a certain property, out of 16, is present or not. Our typical query select 300 "random" rows (could be located in different blocks) from the table based on another column+index, and then filters them down to ~50 based on this t

Re: [PERFORM] Vacuuming technique doubt

2009-06-01 Thread Greg Smith
On Mon, 1 Jun 2009, S Arvind wrote: Having a doubt, we want to vacuum and reindex some 50 most used tables daily on specific time. Is it best to have a function in postgres and call it in cron or is there any other good way to do the two process for specified tables at specified time? If you

Re: [PERFORM] degenerate performance on one server of 3

2009-06-01 Thread Tom Lane
Erik Aronesty writes: > but why wasn't autovac enough to reclaim at least *most* of the space? Autovac isn't meant to reclaim major amounts of bloat; it's more in the line of trying to prevent it from happening in the first place. To reclaim bloat it would have to execute VACUUM FULL, or some ot

Re: [PERFORM] Postgresql cache (memory) performance + how to warm up the cache

2009-06-01 Thread Greg Smith
On Mon, 1 Jun 2009, Shaul Dar wrote: 1. At any given time how can I check what portion (%) of specific tables and indexes is cached in memory? This is a bit tricky. PostgreSQL caches information in its shared_buffers cache, and you can get visibility into that if you install the contrib/pg

Re: [PERFORM] Vacuuming technique doubt

2009-06-01 Thread David Rees
On Sun, May 31, 2009 at 10:26 PM, S Arvind wrote: > Having a doubt, we want to vacuum and reindex some 50 most used tables daily > on specific time. Is it best to have a function in postgres and call it in > cron or is there any other good way to do the two process for specified > tables at specif

[PERFORM] Postgresql cache (memory) performance + how to warm up the cache

2009-06-01 Thread Shaul Dar
I have a DB table with 25M rows, ~3K each (i.e. ~75GB), that together with multiple indexes I use (an additional 15-20GB) will not fit entirely in memory (64GB on machine). A typical query locates 300 rows thru an index, optionally filters them down to ~50-300 rows using other indexes, finally fetc