Re: [PERFORM] Group commit and commit delay/siblings

2010-12-05 Thread Jignesh Shah
On Mon, Dec 6, 2010 at 12:35 PM, Greg Smith wrote: > Jignesh Shah wrote: >> >> The commit_siblings = 5 basically checks that it sleeps only when that >> many backends are active. This I think is a very expensive check and I >> would rather make commit_siblings=0 (which the current code does not >>

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/5/10 12:14 PM, Tom Lane wrote: > I wrote: >> You could get rid of the memory growth, at the cost of a lot of >> tree-copying, by doing each child plan step in a discardable memory >> context. I'm not sure that'd be a win for normal sizes of inheritance >> trees though --- you'd need to copy

Re: [PERFORM] Group commit and commit delay/siblings

2010-12-05 Thread Greg Smith
Jignesh Shah wrote: The commit_siblings = 5 basically checks that it sleeps only when that many backends are active. This I think is a very expensive check and I would rather make commit_siblings=0 (which the current code does not support.. it only supports minimum of 1) I just posted a message

Re: [PERFORM] Group commit and commit delay/siblings

2010-12-05 Thread Jignesh Shah
On Mon, Dec 6, 2010 at 10:47 AM, Rob Wultsch wrote: > On Sun, Dec 5, 2010 at 7:30 PM, Jignesh Shah wrote: >> The commit_siblings = 5 basically checks that it sleeps only when that >> many backends are active. This I think is a very expensive check and I >> would rather make commit_siblings=0 (whi

Re: [PERFORM] Group commit and commit delay/siblings

2010-12-05 Thread Rob Wultsch
On Sun, Dec 5, 2010 at 7:30 PM, Jignesh Shah wrote: > The commit_siblings = 5 basically checks that it sleeps only when that > many backends are active. This I think is a very expensive check and I > would rather make commit_siblings=0 (which the current code does not > support.. it only supports

Re: [PERFORM] Group commit and commit delay/siblings

2010-12-05 Thread Jignesh Shah
On Mon, Dec 6, 2010 at 4:40 AM, Rob Wultsch wrote: > Manual: > http://www.postgresql.org/docs/9.0/static/runtime-config-wal.html#RUNTIME-CONFIG-WAL-SETTINGS > Recent discussion: > http://www.facebook.com/notes/mysql-at-facebook/group-commit-in-postgresql/465781235932 > > It is my understanding th

[PERFORM] Group commit and commit delay/siblings

2010-12-05 Thread Rob Wultsch
Manual: http://www.postgresql.org/docs/9.0/static/runtime-config-wal.html#RUNTIME-CONFIG-WAL-SETTINGS Recent discussion: http://www.facebook.com/notes/mysql-at-facebook/group-commit-in-postgresql/465781235932 It is my understanding that group commit in PG works without the commit_delay or commit_

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread Tom Lane
I wrote: > You could get rid of the memory growth, at the cost of a lot of > tree-copying, by doing each child plan step in a discardable memory > context. I'm not sure that'd be a win for normal sizes of inheritance > trees though --- you'd need to copy the querytree in and then copy the > result

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread Tom Lane
John Papandriopoulos writes: > On 12/4/10 2:40 PM, Tom Lane wrote: >> [ pokes at that for a bit ... ] Ah, I had forgotten that UPDATE/DELETE >> go through inheritance_planner() while SELECT doesn't. And >> inheritance_planner() makes a copy of the querytree, including the >> already-expanded ran

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/4/10 3:19 PM, Mladen Gogala wrote: > Tom Lane wrote: >> Partitioning using these techniques will work well with up to >> perhaps a hundred partitions; don't try to use many thousands of >> partitions. >> regards, tom lane > Hmmm, what happens if I need 10 years of data, in monthly partitions?

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/4/10 2:40 PM, Tom Lane wrote: > [ pokes at that for a bit ... ] Ah, I had forgotten that UPDATE/DELETE > go through inheritance_planner() while SELECT doesn't. And > inheritance_planner() makes a copy of the querytree, including the > already-expanded range table, for each target relation.

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/4/10 3:38 PM, Jochen Erwied wrote: > Sunday, December 5, 2010, 12:19:29 AM you wrote: > >> Hmmm, what happens if I need 10 years of data, in monthly partitions? It >> would be 120 partitions. Can you please elaborate on that limitation? >> Any plans on lifting that restriction? > > I'm runn