Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Carey
On 5/28/09 6:54 PM, "Greg Smith" wrote: > 2) You have very new hardware and a very old kernel. Once you've done the > above, if you're still not happy with performance, at that point you > should consider using a newer one. It's fairly simple to build a Linux > kernel using the same basic kern

Re: [PERFORM] autovacuum hung?

2009-05-29 Thread Brian Cox
Alvaro Herrera [alvhe...@commandprompt.com] wrote: What's vacuum_cost_delay? #vacuum_cost_delay = 0 # 0-1000 milliseconds #vacuum_cost_page_hit = 1 # 0-1 credits #vacuum_cost_page_miss = 10 # 0-1 credits #vacuum_cost_page_dirty = 20#

Re: [PERFORM] autovacuum hung?

2009-05-29 Thread Alvaro Herrera
Brian Cox wrote: > autovacuum has been running on 2 tables for > 5 hours. There tables are > not huge (see below). For the past ~1 hour, I've shut off all other > activity on this database. The other table being vacuumed has more rows > (1897810). Anyone have any ideas about why this is takin

[PERFORM] autovacuum hung?

2009-05-29 Thread Brian Cox
autovacuum has been running on 2 tables for > 5 hours. There tables are not huge (see below). For the past ~1 hour, I've shut off all other activity on this database. The other table being vacuumed has more rows (1897810). Anyone have any ideas about why this is taking so long? Thanks, Brian

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Robert Haas
On Fri, May 29, 2009 at 5:57 PM, Anne Rosset wrote: > Robert Haas wrote: > >> On Thu, May 28, 2009 at 6:46 PM, Anne Rosset wrote: >> >>> >>>                                                ->  Index Scan using >>> item_pk on item  (cost=0.00..176865.31 rows=97498 width=88) (actual >>> time=117.304

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Anne Rosset
Robert Haas wrote: On Thu, May 28, 2009 at 6:46 PM, Anne Rosset wrote: -> Index Scan using item_pk on item (cost=0.00..176865.31 rows=97498 width=88) (actual time=117.304..2405.060 rows=71 loops=1)

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Robert Haas
> When the query plan takes a wrong path, is it possible that it is because > statistics have not been run or updated? Yes. If you are not using autovacuum, you need to ANALYZE regularly, or bad things will happen to you. ...Robert -- Sent via pgsql-performance mailing list (pgsql-performance@

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Robert Haas
On Thu, May 28, 2009 at 6:46 PM, Anne Rosset wrote: >                                                  ->  Index Scan using > item_pk on item  (cost=0.00..176865.31 rows=97498 width=88) (actual > time=117.304..2405.060 rows=71 loops=1) >                                                        Filte

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Anne Rosset
Dave Dutcher wrote: From: Anne Rosset Subject: Re: [PERFORM] Unexpected query plan results Thank Dave. We are using postgresql-server-8.2.4-1PGDG and have work-mem set to 20MB. What value would you advise? thanks, Anne Work-mem is kind of tricky because the right settin

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Mead
On Fri, May 29, 2009 at 3:45 PM, Fabrix wrote: > > Which is better and more complete, which have more features? > What you recommend? pgbouncer or pgpool? > >> In your case, where you're looking to just get the connection overhead off of the machine, pgBouncer is probably going to be more effi

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Fabrix
2009/5/29 Scott Mead > 2009/5/29 Greg Smith > >> On Fri, 29 May 2009, Grzegorz Ja?kiewicz wrote: >> >> if it is implemented somewhere else better, shouldn't that make it >>> obvious that postgresql should solve it internally ? >>> >> >> Opening a database connection has some overhead to it that

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Scott Mead
On Fri, May 29, 2009 at 1:30 PM, Dave Dutcher wrote: > > From: Anne Rosset > > Subject: Re: [PERFORM] Unexpected query plan results > > > > > > > > Thank Dave. We are using postgresql-server-8.2.4-1PGDG and > > have work-mem set to 20MB. > > What value would you advise? > > thanks, > > > > Anne >

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Marlowe
On Fri, May 29, 2009 at 12:20 PM, Scott Mead wrote: >  This sounds like a dirty plug (sorry sorry sorry, it's for informative > purposes only)... (Commercial applications mentioned deleted for brevity.) Just sounded like useful information to me. I'm not anti commercial, just anti-marketing spe

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Mead
2009/5/29 Greg Smith > On Fri, 29 May 2009, Grzegorz Ja?kiewicz wrote: > > if it is implemented somewhere else better, shouldn't that make it >> obvious that postgresql should solve it internally ? >> > > Opening a database connection has some overhead to it that can't go away > without losing *

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Greg Smith
On Fri, 29 May 2009, Fabrix wrote: In this application is not closing the connection, the development team is makeing the change for close the connection after getting the job done. So most connections are in idle state.  How much would this help? Does this could be the real problem? Ah, now

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Fabrix
2009/5/28 Greg Smith > On Thu, 28 May 2009, Flavio Henrique Araque Gurgel wrote: > > It is 2.6.24 We had to apply the kswapd patch also. It's important >> specially if you see your system % going as high as 99% in top and loosing >> the machine's control. I have read something about 2.6.28 had t

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Greg Smith
On Fri, 29 May 2009, Grzegorz Ja?kiewicz wrote: if it is implemented somewhere else better, shouldn't that make it obvious that postgresql should solve it internally ? Opening a database connection has some overhead to it that can't go away without losing *something* in the process that you w

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Dave Dutcher
> From: Anne Rosset > Subject: Re: [PERFORM] Unexpected query plan results > > > > > Thank Dave. We are using postgresql-server-8.2.4-1PGDG and > have work-mem set to 20MB. > What value would you advise? > thanks, > > Anne Work-mem is kind of tricky because the right setting depends on how mu

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Anne Rosset
Dave Dutcher wrote: From: Anne Rosset Subject: [PERFORM] Unexpected query plan results Hi, We have one query which has a left join. If we run this query without the left join, it runs slower than with the left join. [snip] I am having a hard time to understand why the query runs fast

Re: [PERFORM] Unexpected query plan results

2009-05-29 Thread Dave Dutcher
> From: Anne Rosset > Subject: [PERFORM] Unexpected query plan results > > Hi, > We have one query which has a left join. If we run this query without > the left join, it runs slower than with the left join. [snip] > I am having a hard time to understand why the query runs > faster with the > l

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Marlowe
2009/5/29 Grzegorz Jaśkiewicz : > damn I agree with you Scott. I wish I had enough cash here to employ > Tom and other pg magicians to improve performance for all of us ;) > > Thing is tho, postgresql is mostly used by companies, that either > don't have that sort of cash, but still like to get the

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Grzegorz Jaśkiewicz
damn I agree with you Scott. I wish I had enough cash here to employ Tom and other pg magicians to improve performance for all of us ;) Thing is tho, postgresql is mostly used by companies, that either don't have that sort of cash, but still like to get the performance, or companies that have 'why

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Marlowe
2009/5/29 Grzegorz Jaśkiewicz : > 2009/5/29 Scott Marlowe : > >> >> Both Oracle and PostgreSQL have fairly heavy backend processes, and >> running hundreds of them on either database is a mistake.    Sure, >> Oracle can handle more transactions and scales a bit better, but no >> one wants to have t

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Grzegorz Jaśkiewicz
2009/5/29 Scott Marlowe : > > Both Oracle and PostgreSQL have fairly heavy backend processes, and > running hundreds of them on either database is a mistake.    Sure, > Oracle can handle more transactions and scales a bit better, but no > one wants to have to buy a 128 way E15K to handle the load

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Marlowe
2009/5/29 Grzegorz Jaśkiewicz : > 2009/5/29 Scott Marlowe : > >>> if it is implemented somewhere else better, shouldn't that make it >>> obvious that postgresql should solve it internally ? It is really >>> annoying to hear all the time that you should add additional path of >>> execution to alread

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Grzegorz Jaśkiewicz
2009/5/29 Scott Marlowe : >> if it is implemented somewhere else better, shouldn't that make it >> obvious that postgresql should solve it internally ? It is really >> annoying to hear all the time that you should add additional path of >> execution to already complex stack, and rely on more code

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Marlowe
2009/5/29 Grzegorz Jaśkiewicz : > On Fri, May 29, 2009 at 2:54 AM, Greg Smith wrote: > >>  The PostgreSQL connection handler is known to be bad at handling high >> connection loads compared to the popular pooling projects, so you really >> shouldn't throw this problem at it. While kernel problems

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Scott Marlowe
2009/5/29 Grzegorz Jaśkiewicz : > On Fri, May 29, 2009 at 2:54 AM, Greg Smith wrote: > >>  The PostgreSQL connection handler is known to be bad at handling high >> connection loads compared to the popular pooling projects, so you really >> shouldn't throw this problem at it. While kernel problems

Re: [PERFORM] Scalability in postgres

2009-05-29 Thread Grzegorz Jaśkiewicz
On Fri, May 29, 2009 at 2:54 AM, Greg Smith wrote: >  The PostgreSQL connection handler is known to be bad at handling high > connection loads compared to the popular pooling projects, so you really > shouldn't throw this problem at it. While kernel problems stack on top of > that, you really sho