Re: [PERFORM] using a stored proc that returns a result set in a complex SQL stmt

2007-10-16 Thread Marcin Stępnicki
On 10/16/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > > The only reason I am going down this road is because of the difficulty of > > using temp tables ( i.e. needing to execute a SQL string). Does anyone know > > if this requirement may be removed in the near future? > > I don't understand

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Joshua D. Drake
On Tue, 16 Oct 2007 17:03:39 -0700 (PDT) Brian Herlihy <[EMAIL PROTECTED]> wrote: > Would it make sense to show the FSM stats for individual table > vaccums as well? I'm wondering if the reason they aren't shown is > because it wouldn't be useful or isn't practical, or just that it > hasn't been

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Tom Lane
Brian Herlihy <[EMAIL PROTECTED]> writes: > Would it make sense to show the FSM stats for individual table vaccums as w= > ell? I'm wondering if the reason they aren't shown is because it wouldn't = > be useful or isn't practical, or just that it hasn't been done. It was intentionally omitted in

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Brian Herlihy
Would it make sense to show the FSM stats for individual table vaccums as well? I'm wondering if the reason they aren't shown is because it wouldn't be useful or isn't practical, or just that it hasn't been done. Brian - Original Message From: Tom Lane <[EMAIL PROTECTED]> If you're t

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Tom Lane
"Scott Marlowe" <[EMAIL PROTECTED]> writes: > On 10/16/07, St=E9phane Schildknecht > <[EMAIL PROTECTED]> wrote: >> That's also why I am so disappointed vacuum doesn't give me these 4 >> hints lines. > What kind of error, or output, does it give you at the end? Any hint > as to why they're missing

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread 李彦 Ian Li
How about shared memory related settings of your kernel? ie. kernel.shmall, kernel.shmmax . Have a check with sysctl, maybe they should be raised: http://www.postgresql.org/docs/8.2/interactive/kernel-resources.html Regards Jason Lustig wrote: I ran "ulimit -a" for the postgres user, and her

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Scott Marlowe
On 10/16/07, Stéphane Schildknecht <[EMAIL PROTECTED]> wrote: > Tom Lane a écrit : > > =?ISO-8859-1?Q?St=E9phane_Schildknecht?= <[EMAIL PROTECTED]> writes: > > > >> For some times, we have a vacuuming process on a specific table that > >> goes slower and slower. In fact, it took some 3 minutes a mo

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Tom Lane
I wrote: > ... The weird thing about this > is why the large maintenance_work_mem works for a regular session and > not for autovacuum. There really shouldn't be much difference in the > maximum workable setting for the two cases, AFAICS. After re-reading the thread I realized that the OP is comp

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Tom Lane
=?ISO-8859-1?Q?St=E9phane_Schildknecht?= <[EMAIL PROTECTED]> writes: > Tom Lane a écrit : >> The real question is how often do rows get updated? I suspect you >> probably need to vacuum this table more than once a day. > To be honest, I suspect it too. But, I have been told by people using > that

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Stéphane Schildknecht
Tom Lane a écrit : > =?ISO-8859-1?Q?St=E9phane_Schildknecht?= <[EMAIL PROTECTED]> writes: > >> For some times, we have a vacuuming process on a specific table that >> goes slower and slower. In fact, it took some 3 minutes a month ago, and >> now it take almost 20 minutes. But, if one day it tak

Re: [PERFORM] using a stored proc that returns a result set in a complex SQL stmt

2007-10-16 Thread Heikki Linnakangas
chrisj wrote: > I am trying to decide between using a temporary table or a stored proc that > returns a result set to solve a fairly complex problem, and was wondering if > Postres, when it sees a stored proc reference in a SQL, is smart enough to, > behind the scenes, create a temporary table with

[PERFORM] using a stored proc that returns a result set in a complex SQL stmt

2007-10-16 Thread chrisj
Hi, I am trying to decide between using a temporary table or a stored proc that returns a result set to solve a fairly complex problem, and was wondering if Postres, when it sees a stored proc reference in a SQL, is smart enough to, behind the scenes, create a temporary table with the results of

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Tom Lane
Richard Huxton <[EMAIL PROTECTED]> writes: > Hmm - odd that you're not getting any connection details. Not really; the autovacuum process doesn't have any connection, so those log_line_prefix fields will be left empty. The weird thing about this is why the large maintenance_work_mem works for a r

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Rodrigo Gonzalez
Jason Lustig escribió: On Oct 16, 2007, at 10:22 AM, Richard Huxton wrote: Add some lines to /etc/security/limits.conf to increase them. Sorry for being somewhat of a linux novice -- but what is the best way to do this? It doesn't seem to provide matching options from ulimit to the limits.c

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Jason Lustig
On Oct 16, 2007, at 10:22 AM, Richard Huxton wrote: Add some lines to /etc/security/limits.conf to increase them. Sorry for being somewhat of a linux novice -- but what is the best way to do this? It doesn't seem to provide matching options from ulimit to the limits.conf file. Thanks, Ja

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Mark Lewis
On Tue, 2007-10-16 at 10:14 -0400, Jason Lustig wrote: > I ran "ulimit -a" for the postgres user, and here's what I got: ... > max memory size (kbytes, -m) 20 > open files (-n) 100 > max user processes (-u) 100 > virtual memory (kbytes, -v) 200

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Tom Lane
=?ISO-8859-1?Q?St=E9phane_Schildknecht?= <[EMAIL PROTECTED]> writes: > For some times, we have a vacuuming process on a specific table that > goes slower and slower. In fact, it took some 3 minutes a month ago, and > now it take almost 20 minutes. But, if one day it take so many time, it > is possi

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Scott Marlowe
On 10/16/07, Jason Lustig <[EMAIL PROTECTED]> wrote: > I ran "ulimit -a" for the postgres user, and here's what I got: > > core file size (blocks, -c) 20 > data seg size (kbytes, -d) 20 > max nice(-e) 0 > file size (blocks, -f) unlimi

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Richard Huxton
Jason Lustig wrote: I ran "ulimit -a" for the postgres user, and here's what I got: max memory size (kbytes, -m) 20 virtual memory (kbytes, -v) 20 There you go - you probably are exceeding these. Add some lines to /etc/security/limits.conf to increase them. -- Ri

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Jason Lustig
I ran "ulimit -a" for the postgres user, and here's what I got: core file size (blocks, -c) 20 data seg size (kbytes, -d) 20 max nice(-e) 0 file size (blocks, -f) unlimited pending signals (-i) 32635 max locked memor

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Scott Marlowe
On 10/16/07, Jason Lustig <[EMAIL PROTECTED]> wrote: > Looking at my free memory (from TOP) I find > > Mem: 2062364k total, 1846696k used, 215668k free, 223324k buffers > Swap: 2104496k total, 160k used, 2104336k free, 928216k cached > > So I don't think that I'm running out of mem

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Richard Huxton
Jason Lustig wrote: I lowered the maintenance_work_mem to 50MB and am still getting the same errors: Oct 16 09:26:57 [16402]: [1-1] user=,db= ERROR: out of memory Oct 16 09:26:57 [16402]: [1-2] user=,db= DETAIL: Failed on request of size 52428798. Oct 16 09:27:57 [16421]: [1-1] user=,db=

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Jason Lustig
I lowered the maintenance_work_mem to 50MB and am still getting the same errors: Oct 16 09:26:57 [16402]: [1-1] user=,db= ERROR: out of memory Oct 16 09:26:57 [16402]: [1-2] user=,db= DETAIL: Failed on request of size 52428798. Oct 16 09:27:57 [16421]: [1-1] user=,db= ERROR: out of mem

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Richard Huxton
Jason Lustig wrote: There isn't any more error message than this... it simply repeats every minute or so, which is really quite strange. And the user & db is really blank in the log, I didn't trim it (if I did I would have replaced it with dummy values). Hmm - odd that you're not getting any

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Jason Lustig
There isn't any more error message than this... it simply repeats every minute or so, which is really quite strange. And the user & db is really blank in the log, I didn't trim it (if I did I would have replaced it with dummy values). I'm using pg 8.2.4 on Linux 2.6.15. Jason -- Jason Lus

Re: [PERFORM] Autovacuum running out of memory

2007-10-16 Thread Richard Huxton
Not really a performance question, but... Jason Lustig wrote: Whenever I turn on Autovacuum on my database, I get a ton of error messages like this in my Postgres log: Oct 16 06:43:47 [2897]: [1-1] user=,db= ERROR: out of memory Oct 16 06:43:47 [2897]: [1-2] user=,db= DETAIL: Failed on req

[PERFORM] Autovacuum running out of memory

2007-10-16 Thread Jason Lustig
Whenever I turn on Autovacuum on my database, I get a ton of error messages like this in my Postgres log: Oct 16 06:43:47 [2897]: [1-1] user=,db= ERROR: out of memory Oct 16 06:43:47 [2897]: [1-2] user=,db= DETAIL: Failed on request of size 524287998. It always fails on the same request

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Stéphane Schildknecht
Heikki Linnakangas a écrit : > Stéphane Schildknecht wrote: > >> I wonder vacuum verbose would tell me if fsm parameters were not too >> badly configured, but I can't get the 4 last lines of the output... >> > > Why not? > I would like to know... Seems like vacuum does not want me to se

Re: [PERFORM] Vacuum goes worse

2007-10-16 Thread Heikki Linnakangas
Stéphane Schildknecht wrote: > I wonder vacuum verbose would tell me if fsm parameters were not too > badly configured, but I can't get the 4 last lines of the output... Why not? > Whats's more, I wonder what we could monitor to get some explanation of > the recent time increase, and then have a

[PERFORM] Vacuum goes worse

2007-10-16 Thread Stéphane Schildknecht
Hi, For some times, we have a vacuuming process on a specific table that goes slower and slower. In fact, it took some 3 minutes a month ago, and now it take almost 20 minutes. But, if one day it take so many time, it is possible that on the day after it will only take 4 minutes... I know the tab