Re: [HACKERS] pg_autovacuum -> pg_class.reloptions?

2007-07-06 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > A long time ago, Tom proposed moving the pg_autovacuum settings into > reloptions. I know it's really late in the devel cycle but I wonder if > such a move would be acceptable at this time? I think it's too late to be considering essentially-cosmetic c

[HACKERS] pg_autovacuum -> pg_class.reloptions?

2007-07-06 Thread Alvaro Herrera
Hi, A long time ago, Tom proposed moving the pg_autovacuum settings into reloptions. I know it's really late in the devel cycle but I wonder if such a move would be acceptable at this time? I feel it would be a good move, for example per http://thread.gmane.org/gmane.comp.db.postgresql.general/9

Re: [HACKERS] pg_autovacuum startup from /etc/rc fails after system crash

2005-09-26 Thread Jim C. Nasby
As a work-around, you can use the scripts at http://cvs.distributed.net/viewcvs.cgi/stats-sql/tools/ On Thu, Sep 22, 2005 at 02:16:58PM -0400, Jonathan Beit-Aharon wrote: > > Hi, > I'm not a member of this list, so please CC me on responses and > discussion. > After a system crash PostgreSQL star

[HACKERS] pg_autovacuum startup from /etc/rc fails after system crash

2005-09-22 Thread Jonathan Beit-Aharon
Hi, I'm not a member of this list, so please CC me on responses and discussion. After a system crash PostgreSQL startup is slow as the database recovers.  So the db_connect() call from pg_autovacuum terminates as soon as it tries to connect to "template1". Looking at the README file, I find t

Re: [HACKERS] pg_autovacuum settings not saved on dump

2005-09-20 Thread Jim C. Nasby
On Thu, Sep 15, 2005 at 11:24:23PM -0400, Tom Lane wrote: > Robert Treat <[EMAIL PROTECTED]> writes: > > On Thursday 15 September 2005 15:41, Alvaro Herrera wrote: > >> Robert Treat reminded me the other day that we don't currently save > >> pg_autovacuum settings on pg_dump. > > > ISTM this is a

Re: [HACKERS] pg_autovacuum settings not saved on dump

2005-09-15 Thread Tom Lane
Robert Treat <[EMAIL PROTECTED]> writes: > On Thursday 15 September 2005 15:41, Alvaro Herrera wrote: >> Robert Treat reminded me the other day that we don't currently save >> pg_autovacuum settings on pg_dump. > ISTM this is a backwards incompatibility with previous installations that > might h

Re: [HACKERS] pg_autovacuum settings not saved on dump

2005-09-15 Thread Robert Treat
On Thursday 15 September 2005 15:41, Alvaro Herrera wrote: > Hi, > > Robert Treat reminded me the other day that we don't currently save > pg_autovacuum settings on pg_dump. > Does it merit a mention on the release notes? > ISTM this is a backwards incompatibility with previous installations th

Re: [HACKERS] pg_autovacuum settings not saved on dump

2005-09-15 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Expected or not, fact is it's not user friendly. We should at least > document this somewhere so users can take care of it by themselves. Not > sure where does it belong though. The autovacuum section, the backup > section? Does it merit a mention on

[HACKERS] pg_autovacuum settings not saved on dump

2005-09-15 Thread Alvaro Herrera
Hi, Robert Treat reminded me the other day that we don't currently save pg_autovacuum settings on pg_dump. This is expected, because we don't want to dump pg_autovacuum as a regular table, or it would force us to accept loading that forever; nor we do have ALTER TABLE commands to do it on a highe

RES: [HACKERS] Pg_autovacuum on FreeBSD

2005-07-08 Thread Rodrigo Moreno
Kings-Lynne Enviada em: quinta-feira, 7 de julho de 2005 23:26 Para: Rodrigo Moreno Cc: pgsql-hackers@postgresql.org Assunto: Re: [HACKERS] Pg_autovacuum on FreeBSD > The pg_autovacuum on FreeBSD and pg 803 is not working. Just do > nothing, no log, nothing in screen, no daemonize. > >

Re: [HACKERS] Pg_autovacuum on FreeBSD

2005-07-07 Thread Christopher Kings-Lynne
The pg_autovacuum on FreeBSD and pg 803 is not working. Just do nothing, no log, nothing in screen, no daemonize. It was ok on pg746. Could some one help me ? They both work fine for me on my test box... Are you aware that they change the port? You need to put postgresql="YES" in your /etc/

Re: [HACKERS] Pg_autovacuum on FreeBSD

2005-07-07 Thread Mark Kirkwood
Rodrigo Moreno wrote: Hi All, The pg_autovacuum on FreeBSD and pg 803 is not working. Just do nothing, no log, nothing in screen, no daemonize. It was ok on pg746. Could some one help me ? What version of FreeBSD are you running? Mark ---(end of broadcast)-

[HACKERS] Pg_autovacuum on FreeBSD

2005-07-07 Thread Rodrigo Moreno
Hi All, The pg_autovacuum on FreeBSD and pg 803 is not working. Just do nothing, no log, nothing in screen, no daemonize. It was ok on pg746. Could some one help me ? Best Regards Rodrigo Moreno ---(end of broadcast)--- TIP 6: Have you searched

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Mark Wong
On Wed, Jan 12, 2005 at 09:17:33PM -0500, Tom Lane wrote: > I notice that the backend seems to have been using some nonstandard C > code: > > Error while reading shared library symbols: > /home/markw/dbt2/storedproc/pgsql/c/../../../storedproc/pgsql/c/funcs.so: No > such file or directory. > > W

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Ok, well I got a core dump with 8.0rc4, but I'm not sure if it's > exactly the same problem. I have the postgres binary and the core > here: > http://developer.osdl.org/markw/pgsql/core/2files.tar.bz2 > But it's for ia64, if you got one. Poking around

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Mark Wong
On Tue, Dec 21, 2004 at 05:56:47PM -0500, Tom Lane wrote: > If you want to track it yourself, please change those elog(ERROR)s to > elog(PANIC) so that they'll generate core dumps, then build with > --enable-debug if you didn't already (--enable-cassert would be good too) > and get a debugger stack

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Matthew T. O'Connor
Mark Wong wrote: The overall throughput is better for a run like this: http://www.osdl.org/projects/dbt2dev/results/dev4-010/207/ A drop from 3865 to 2679 (31%) by just adding pg_autovacuum. That's what I meant by "not good". :) I would agree that is "not good" :-) It sounds like pg_au

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > I was going to try Matthew's suggestion of turning up the debug on > pg_autovacuum, unless you don't that'll help find the cause. It won't help --- this is a backend-internal bug of some kind. regards, tom lane -

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
On Tue, Dec 21, 2004 at 05:56:47PM -0500, Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > On Tue, Dec 21, 2004 at 02:23:41PM -0500, Tom Lane wrote: > >> Mark Wong <[EMAIL PROTECTED]> writes: > >>> [2004-12-20 15:48:18 PST] The error is [ERROR: failed to > >>> re-find parent k

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > On Tue, Dec 21, 2004 at 02:23:41PM -0500, Tom Lane wrote: >> Mark Wong <[EMAIL PROTECTED]> writes: >>> [2004-12-20 15:48:18 PST] The error is [ERROR: failed to re-find >>> parent key in "pk_district" >> >> Yikes. Is this reproducible? > Yes, and

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
The overall throughput is better for a run like this: http://www.osdl.org/projects/dbt2dev/results/dev4-010/207/ A drop from 3865 to 2679 (31%) by just adding pg_autovacuum. That's what I meant by "not good". :) I'll start with the additional debug messages, with 8.0rc2, before I start c

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
On Tue, Dec 21, 2004 at 02:23:41PM -0500, Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > [2004-12-20 15:48:18 PST] The error is [ERROR: failed to re-find > > parent key in "pk_district" > > ] > > Yikes. Is this reproducible? > > regards, tom lane Ye

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Matthew T. O'Connor
Mark Wong wrote: After all this time I finally got around to vacuuming the database with dbt2 with pg_autovacuum. :) http://www.osdl.org/projects/dbt2dev/results/dev4-010/215/ Thanks! Doesn't look so good though, probably because I'm not using optimal settings with pg_autovacuum. So far I have

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > [2004-12-20 15:48:18 PST] The error is [ERROR: failed to re-find > parent key in "pk_district" > ] Yikes. Is this reproducible? regards, tom lane ---(end of broadcast)--- T

[HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
After all this time I finally got around to vacuuming the database with dbt2 with pg_autovacuum. :) http://www.osdl.org/projects/dbt2dev/results/dev4-010/215/ Doesn't look so good though, probably because I'm not using optimal settings with pg_autovacuum. So far I have only tried the defa

Re: [HACKERS] pg_autovacuum

2004-09-24 Thread Matthew T. O'Connor
pg_autovacuum just writes to standard out unless you specify a log file on the command line. See pg_autovacuum -h for details. Matthew On Wed, 2004-09-22 at 03:29, Iulia Pacurar wrote: Hi! I run pg_autovacuum: ./pg_autovacuum -D but then I cannot find pg_autovacuum.log file. Where shoud I look f

Re: [HACKERS] pg_autovacuum

2004-09-23 Thread Thomas F . O'Connell
You also need to use -L to specify a location for the log file. By default pg_autovacuum just logs to STDERR, so if you daemonize the process (via -D), you won't be able to recover the output easily unless you explicitly select a log file location. -tfo On Sep 22, 2004, at 2:29 AM, Iulia Pacura

[HACKERS] pg_autovacuum

2004-09-23 Thread Iulia Pacurar
Hi! I run pg_autovacuum: ./pg_autovacuum -D but then I cannot find pg_autovacuum.log file. Where shoud I look for it? Thank you. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEm

Re: [HACKERS] pg_autovacuum and v8.0

2004-09-09 Thread Hervé Piedvache
OK thanks ! Le Jeudi 9 Septembre 2004 18:14, Thomas F.O'Connell a écrit : > pg_autovacuum will still be in contrib as of 8.0. It did not make > integration with the core distribution. > > -tfo > > On Sep 9, 2004, at 11:09 AM, Hervé Piedvache wrote: > > Hi, > > > > I was thinking that new things wi

Re: [HACKERS] pg_autovacuum and v8.0

2004-09-09 Thread Thomas F . O'Connell
pg_autovacuum will still be in contrib as of 8.0. It did not make integration with the core distribution. -tfo On Sep 9, 2004, at 11:09 AM, Hervé Piedvache wrote: Hi, I was thinking that new things will appear in v8.0 about pg_autovacuum ?? But I find nothing new in README and/or Version Histor

[HACKERS] pg_autovacuum and v8.0

2004-09-09 Thread Hervé Piedvache
Hi, I was thinking that new things will appear in v8.0 about pg_autovacuum ?? But I find nothing new in README and/or Version History Any help ? Regards, -- Hervé Piedvache Elma Ingénierie Informatique 6 rue du Faubourg Saint-Honoré F-75008 - Paris - France Pho. 33-144949901 Fax. 33-1449

Re: [HACKERS] pg_autovacuum Win32 Service Code

2004-08-11 Thread Dave Page
> -Original Message- > From: Matthew T. O'Connor [mailto:[EMAIL PROTECTED] > Sent: 10 August 2004 21:41 > To: Tom Lane > Cc: Dave Page; PostgreSQL-development; > [EMAIL PROTECTED]; Bruce Momjian > Subject: Re: [HACKERS] pg_autovacuum Win32 Service Code

Re: [HACKERS] pg_autovacuum Win32 Service Code

2004-08-10 Thread Matthew T. O'Connor
Tom Lane wrote: "Dave Page" <[EMAIL PROTECTED]> writes: Some time ago I posted a patch to allow pg_autovacuum to be run as a service on Windows, on the understanding that if pg_autovacuum didn't make it into the backend, it would be applied. I followed up Tom's message to Matthew apologising for no

Re: [HACKERS] pg_autovacuum Win32 Service Code

2004-08-10 Thread Bruce Momjian
Tom Lane wrote: > "Dave Page" <[EMAIL PROTECTED]> writes: > > Some time ago I posted a patch to allow pg_autovacuum to be run as a > > service on Windows, on the understanding that if pg_autovacuum didn't > > make it into the backend, it would be applied. I followed up Tom's > > message to Matthew

Re: [HACKERS] pg_autovacuum Win32 Service Code

2004-08-10 Thread Tom Lane
"Dave Page" <[EMAIL PROTECTED]> writes: > Some time ago I posted a patch to allow pg_autovacuum to be run as a > service on Windows, on the understanding that if pg_autovacuum didn't > make it into the backend, it would be applied. I followed up Tom's > message to Matthew apologising for not includ

[HACKERS] pg_autovacuum Win32 Service Code

2004-08-10 Thread Dave Page
Some time ago I posted a patch to allow pg_autovacuum to be run as a service on Windows, on the understanding that if pg_autovacuum didn't make it into the backend, it would be applied. I followed up Tom's message to Matthew apologising for not including his backend integration patches with a reque

[HACKERS] pg_autovacuum Integration

2004-05-28 Thread Matthew T. O'Connor
Since the Feature Freeze is coming on quickly and I have yet to submit a patch that integrated pg_autovacuum into the backend (though I have been working on it), I wanted to see what people thing about a few things. Since we are nearing feature freeze, I know won't complete all the improvements to

Re: [HACKERS] pg_autovacuum fixes

2004-05-26 Thread Bruce Momjian
Patch applied. Thanks. Backpatched to 7.4.X. --- Matthew T. O'Connor wrote: > This weekend I am trying to fix up all known the pg_autovacuum issues > that should be resolved for 7.4.3. I am aware of only two issues: tem

Re: [HACKERS] pg_autovacuum fixes

2004-05-23 Thread Bruce Momjian
Matthew T. O'Connor wrote: > This weekend I am trying to fix up all known the pg_autovacuum issues > that should be resolved for 7.4.3. I am aware of only two issues: temp > table issues, and unchecked send_query() calls, if I am forgetting > something, please let me know. > > 1) temp table issu

Re: [HACKERS] pg_autovacuum fixes

2004-05-23 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches I will try to apply it within the next 48 hours. --- Matthew T. O'Connor wrote: > This weekend

[HACKERS] pg_autovacuum fixes

2004-05-22 Thread Matthew T. O'Connor
This weekend I am trying to fix up all known the pg_autovacuum issues that should be resolved for 7.4.3. I am aware of only two issues: temp table issues, and unchecked send_query() calls, if I am forgetting something, please let me know. 1) temp table issue: I was not able to reproduce the cr

Re: [HACKERS] pg_autovacuum seems to be a neat freak and cleans

2004-05-18 Thread Matthew T. O'Connor
On Tue, 2004-05-18 at 22:21, Brian Hirt wrote: > there might be another similar bug that was fixed in 7.4.2 This bug is fixed, but it didn't make in 7.4.2, it is in CVS (both 7.4 and HEAD). Please grab pg_autovacuum.c and .h from CVS, if that doesn't fix it please let me know. Thanks, Matthew

Re: [HACKERS] pg_autovacuum seems to be a neat freak and cleans way too much

2004-05-18 Thread Brian Hirt
there might be another similar bug that was fixed in 7.4.2 i just doubled checked the 7.4.2 tarball, and it does have this problem. you might want to double check to see if it's fixed in 7.4.3, or i can grab cvs and check it if you like. On May 18, 2004, at 8:06 PM, Bruce Momjian wrote: I think

Re: [HACKERS] pg_autovacuum seems to be a neat freak and cleans way

2004-05-18 Thread Bruce Momjian
I think we already fixed that in 7.4.2. We also have a few bugs still in 7.4.2 and we need to get those fixed soon and release 7.4.3. --- Brian Hirt wrote: > I'm following up on my own email and cross posting to hackers, be

Re: [HACKERS] pg_autovacuum seems to be a neat freak and cleans way too much

2004-05-18 Thread Brian Hirt
I'm following up on my own email and cross posting to hackers, because there is a bug that needs fixed. I spent some more time digging into this, and I found the cause of the problem. reltuples in pg_class is defined as a real, reltuples in pg_autovacuum is defined as an int. the query

Re: [HACKERS] pg_autovacuum next steps

2004-05-17 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: I probably said that wrong, but how do backends get their stats data? They read it out of a flat file that the stats collector rewrites every so often. Ok so that would be easy to do (if we decide we want to) Is that rea

Re: [HACKERS] pg_autovacuum Win32 service patch #2

2004-05-14 Thread Dave Page
> -Original Message- > From: Matthew T. O'Connor [mailto:[EMAIL PROTECTED] > Sent: 13 May 2004 21:40 > To: Dave Page > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: Re: [HACKERS] pg_autovacuum Win32 service patch #2 > > Anyway, not having looked at

Re: [HACKERS] pg_autovacuum Win32 service patch #2

2004-05-13 Thread Matthew T. O'Connor
Dave Page wrote: Any comments/criticisms/gasps of horror at all the win32 code? :-) Sorry for not jumping in sooner but I have been offline for several days. Anyway, not having looked at this at all, how will this be effected when pg_autovacuum is integrated into the backend. I assume that the

[HACKERS] pg_autovacuum Win32 service patch #2

2004-05-09 Thread Dave Page
[Third attempt to send this - dunno where they're all going!] I forgot to CC the start of this to -hackers last time - please see http://archives.postgresql.org/pgsql-hackers-win32/2004-05/msg00034.php for background. Following Magnus' suggestions yesterday I made the following changes: - The ev

Re: [HACKERS] pg_autovacuum misinterpreting reltuples?

2004-05-05 Thread Matthew T. O'Connor
Jeff Boes wrote: We noticed that one of our high-volume insert tables was being vacuumed every time pg_autovacuum woke up. (I"m running it with the default threshold values, and a 900-second sleep cycle.) The table has a few million rows in it. With "debug = 2" on, here's what the pg_autovacuum

[HACKERS] pg_autovacuum misinterpreting reltuples?

2004-05-05 Thread Jeff Boes
We noticed that one of our high-volume insert tables was being vacuumed every time pg_autovacuum woke up. (I"m running it with the default threshold values, and a 900-second sleep cycle.) The table has a few million rows in it. With "debug = 2" on, here's what the pg_autovacuum log reports for

Re: [HACKERS] pg_autovacuum crashes when query fails for temp

2004-04-26 Thread Christopher Browne
[EMAIL PROTECTED] ("Matthew T. O'Connor") wrote: >> Bruce Momjian wrote: >> Should pg_autovacuum be vacuuming temporary tables? > > This is a good question, and I would like some opinions from some other > people more informed than I. > >> Secondly, why would >> a temporary table for another sessio

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-21 Thread Matthew T. O'Connor
On Wednesday 21 April 2004 12:05 am, Christopher Kings-Lynne wrote: > >> No, I have not heard of a 7.4.3 timeline, but we certainly want your > >> eventual fixes in that release. > > > > Right, and along these lines there are a few other pg_autovacuum bugs > > that were fixed just after 7.4.2. > >

Re: [HACKERS] pg_autovacuum crashes when query fails for temp

2004-04-20 Thread Christopher Kings-Lynne
Ok, so I will change pg_autovacuum to explicitly ignore temp tables. Just to be sure, I can do this by avoiding anything found in the pg_temp schemea, or is there a better way? Is it possible that a user could or would put a non-temp table the pg_temp schemea? There's no such thing as the pg_t

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > System Tables: pg_autovacuum treats non-shared system tables just like > any other table. It monitors the activity and vacuums when it deems it > appropriate. As for shared system tables: In user databases they are > only analyzed by pg_auto

Re: [HACKERS] pg_autovacuum crashes when query fails for temp

2004-04-20 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > Just to be sure, I can do this by avoiding anything found in the pg_temp > schemea, or is there a better way? Is it possible that a user could or > would put a non-temp table the pg_temp schemea? The pg_temp_NN schemas are the temp objects, by

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Christopher Kings-Lynne
No, I have not heard of a 7.4.3 timeline, but we certainly want your eventual fixes in that release. Right, and along these lines there are a few other pg_autovacuum bugs that were fixed just after 7.4.2. A rollable log solution would be nice :) Syslog? :) Chris ---(end

Re: [HACKERS] pg_autovacuum crashes when query fails for temp

2004-04-20 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: This is a good question, and I would like some opinions from some other people more informed than I. You *can not* vacuum other sessions' temp tables; you don't have access to the data. (You have no way to get at pages that

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Matthew T. O'Connor
Bruce Momjian wrote: No, I have not heard of a 7.4.3 timeline, but we certainly want your eventual fixes in that release. Right, and along these lines there are a few other pg_autovacuum bugs that were fixed just after 7.4.2. ---(end of broadcast)--

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Matthew T. O'Connor
Christopher Kings-Lynne wrote: Does pg_autovacuum vacuum and analyze system catalog and TOAST tables properly? Properly? I think so, that is to the best of my knowledge which is a bit limited :-) Toast Tables: pg_autovacuum doesn't do anything to toast tables explicitly. I am not aware th

Re: [HACKERS] pg_autovacuum crashes when query fails for temp

2004-04-20 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: >> Bruce Momjian wrote: >> Should pg_autovacuum be vacuuming temporary tables? > This is a good question, and I would like some opinions from some other > people more informed than I. You *can not* vacuum other sessions' temp tables; you don't hav

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Christopher Kings-Lynne
I looked into this and I see a number of cases where pg_autovacuum calls send_query(), but doesn't test for a NULL return from the function. Matthew, would you look into this and submit a patch? Thanks. Does pg_autovacuum vacuum and analyze system catalog and TOAST tables properly? Chris --

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Matthew T. O'Connor
Yeah, I will, I just don't know when. I have been trying to get to this and lots of other pg_autovacuum tasks, but my schedule has been quite crazy as of late. Anyway, this should probably be a pretty simple patch, so I can probably find some time to look at it soon. Any idea on the 7.4.3 releas

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Bruce Momjian
Matthew T. O'Connor wrote: > Yeah, I will, I just don't know when. I have been trying to get to this > and lots of other pg_autovacuum tasks, but my schedule has been quite > crazy as of late. Anyway, this should probably be a pretty simple patch, > so I can probably find some time to look at it

Re: [HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Thomas Swan
Bruce Momjian wrote: I looked into this and I see a number of cases where pg_autovacuum calls send_query(), but doesn't test for a NULL return from the function. Matthew, would you look into this and submit a patch? Thanks. Should pg_autovacuum be vacuuming temporary tables? Secondly, why wo

Re: [HACKERS] pg_autovacuum crashes when query fails for temp

2004-04-20 Thread Matthew T. O'Connor
> Bruce Momjian wrote: > Should pg_autovacuum be vacuuming temporary tables? This is a good question, and I would like some opinions from some other people more informed than I. > Secondly, why would > a temporary table for another session be visible to pg_autovacuum? I > know these may sound li

[HACKERS] pg_autovacuum crashes when query fails for temp tables

2004-04-20 Thread Bruce Momjian
I looked into this and I see a number of cases where pg_autovacuum calls send_query(), but doesn't test for a NULL return from the function. Matthew, would you look into this and submit a patch? Thanks. --- Jeff Boes wrote

Re: [HACKERS] pg_autovacuum

2004-03-23 Thread Matthew T. O'Connor
On Tuesday 23 March 2004 12:32 am, Josh Berkus wrote: > Matt, > > > What I'm thinking is that the VACUUM command could be modified to write > > down some data from the stats system at vacuum time. Once the VACUUM > > command writes this down for itself then pg_autovacuum just uses that > > number

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Alex J. Avriette
On Mon, Mar 22, 2004 at 04:50:57PM -0500, Matthew T. O'Connor wrote: > Could you please explain this better, I don't really understand what the > problem is. If you want pg_autovacuum to perform a vacuum on a table > that has had exactly X updates no matter what, you can just run it with > -V0

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Joe Conway
Matthew T. O'Connor wrote: Joe Conway wrote: This would be *really* nice to have. In my recent case, if pg_autovacuum could work for say 3 minutes, and then back off for 2 minutes or so while the batch transactions hit, it would be ideal. I'm not sure what you are suggesting here. As it stands

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Joe Conway
Matthew T. O'Connor wrote: Could you please explain this better, I don't really understand what the problem is. If you want pg_autovacuum to perform a vacuum on a table that has had exactly X updates no matter what, you can just run it with -V0 -vX (where X is the vacuum threshold) same thing

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
Gavin Sherry wrote: I was initially against the idea of using libpq but its growing on me too. I think it would be good if the core functions of pg_autovacuum: threshold algorithms, connection, issuing commands can be (re?)designed such that not only the backend can link against it but also a str

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
Joe Conway wrote: Matthew T. O'Connor wrote: * Inability to schedule vacuums during off-peak times This would be *really* nice to have. In my recent case, if pg_autovacuum could work for say 3 minutes, and then back off for 2 minutes or so while the batch transactions hit, it would be ideal.

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Joe Conway
Matthew T. O'Connor wrote: * Inability to customize thresholds on a per table basis I ran headlong into this one. IMHO fixing this is critical. * Inability to set default thresholds on a per database basis * Inability to exclude specific databases / tables from pg_autovacuum monitoring These would

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Gavin Sherry
On Mon, 22 Mar 2004, Tom Lane wrote: > "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > > Well this certainly sounds like something that would be easy to do, > > which appeals to me at least as a first cut. Question: Does this mean > > that I lose many of the advantages of being "in the backen

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> There is no "direct pipe connection" to the stats collector, > I probably said that wrong, but how do backends get their stats data? They read it out of a flat file that the stats collector rewrites every so often. > Meanin

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
Tom Lane wrote: If you aren't a backend then you couldn't safely access shared memory, including FSM in particular. I see no reason you couldn't use GUC though. There is no "direct pipe connection" to the stats collector, except in the output direction which is not what you want, so I'm not seei

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > Well this certainly sounds like something that would be easy to do, > which appeals to me at least as a first cut. Question: Does this mean > that I lose many of the advantages of being "in the backend"? That is, > would pg_autovacuum still b

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
Alex J. Avriette wrote: Hi, Matthew. For our uses, we found that pg_autovacuum did not behave as expected with vacuum_threshold set to 0. For our purposes, we have a very good idea of how many tuples need to be slurped up over a given interval, and would like the autovacuum daemon to simply go th

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
Tom Lane wrote: From the point of view of the postmaster a GUC-controlled delay would seem like the best thing. We could discuss having the autovacuum code try to feed back adjustments in the delay, but remember that one of the golden virtues for the postmaster proper is simplicity; that translat

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Gavin Sherry
> > As such, I > > think that a vacuum backend for a specific database should be forked upon > > the first connect. Also, the backend might like to try and workout if > > there are any active backends for its database every so often and if not, > > perform a final vacuum (if necessary) and exit, so

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Alex J. Avriette
On Sun, Mar 21, 2004 at 05:32:59PM -0500, Matthew T. O'Connor wrote: > Lately I have been thinking about the next steps for the pg_autovacuum > daemon. I have written up a document that describes what I'm planning > to do next. Please read the attached and response as I would really > like some

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
On Mon, 2004-03-22 at 10:51, Tom Lane wrote: > > 2.Use a config file. This would require some additional coding to add > > the required parsing, but is possible. > > Personally I like #2. The claim that this requires extra code seems > bogus to me --- when you are working at the C code level,

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
On Mon, 2004-03-22 at 04:23, Karel Zak wrote: > All. It's important do it as backend process. Because libpq has very, > very limited and slow resources for work with backend stuff. Agreed. > The base should be the standard backend with different "main loop" that > will instead socket chec

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
On Mon, 2004-03-22 at 10:58, Tom Lane wrote: > Lots of idle processes sitting around is right out, too. Remember that > each one would eat a backend connection slot. I think we are going to > have to limit this to *one* process at a time. What that probably means > is that we successively launch

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
On Mon, 2004-03-22 at 07:25, Richard Huxton wrote: > On Monday 22 March 2004 03:36, Matthew T. O'Connor wrote: > > 1.Store config data inside a special pg_autovacuum table inside > > existing databases that wants custom settings. > > > > 2.Use a config file. This would require some additional co

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Tom Lane
Gavin Sherry <[EMAIL PROTECTED]> writes: > One point is this: vacuum() assumes that you are running in a fully > fledged backend. There'd be a fair bit of work involved in allowing a > single process to call vacuum() against multiple databases. Make that "it isn't going to happen". > As such, I >

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > There are differing opinions as to the best way to providing these this > feature. The primary debate is where to save the configuration data. I > see three options: > 1.Store config data inside a special pg_autovacuum table inside > existing

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Matthew T. O'Connor
On Mon, 2004-03-22 at 03:36, Gavin Sherry wrote: > One point is this: vacuum() assumes that you are running in a fully > fledged backend. There'd be a fair bit of work involved in allowing a > single process to call vacuum() against multiple databases. I can't imagine we want to do that. > As su

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Jan Wieck
Gavin Sherry wrote: On Sun, 21 Mar 2004, Matthew T. O'Connor wrote: On Sun, 2004-03-21 at 20:31, Christopher Kings-Lynne wrote: > > I think these configuration issues will become a lot easier if you make > > the autovacuum daemon a subprocess of the postmaster (like, say, the > > checkpoint proces

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Richard Huxton
On Monday 22 March 2004 03:36, Matthew T. O'Connor wrote: > 1. Per Database defaults and Per table Thresholds: > > There are differing opinions as to the best way to providing these this > feature. The primary debate is where to save the configuration data. I > see three options: > > 1.Store con

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Karel Zak
On Mon, Mar 22, 2004 at 02:35:37AM -0500, Matthew T. O'Connor wrote: > On Sun, 2004-03-21 at 23:00, Bruce Momjian wrote: > > > > C) Most importantly, I'm not backend hacker. If someone wants to do the > > > > initial work of getting it running as a backend process, I can take it > > > > from there

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Gavin Sherry
> > Ok, thanks for the offer to help, but I think I understated things above > > when I said I'll need a "little" help :-) > > > > I haven't looked at the code but... > > > I have a few big picture questions. Once pg_autovacuum is launched as a > > postmaster sub-process, what changes? That is, c

Re: [HACKERS] pg_autovacuum next steps

2004-03-22 Thread Gavin Sherry
On Mon, 22 Mar 2004, Matthew T. O'Connor wrote: > On Sun, 2004-03-21 at 23:00, Bruce Momjian wrote: > > > > C) Most importantly, I'm not backend hacker. If someone wants to do the > > > > initial work of getting it running as a backend process, I can take it > > > > from there. A while ago, Bruc

Re: [HACKERS] pg_autovacuum next steps

2004-03-21 Thread Bruce Momjian
> > B) Perhaps people like the idea of it being a client app (I don't think > > so.) > > > > I'd like to see it as part of the backend. > > > C) Most importantly, I'm not backend hacker. If someone wants to do the > > initial work of getting it running as a backend process, I can take it > > fro

Re: [HACKERS] pg_autovacuum next steps

2004-03-21 Thread Gavin Sherry
On Sun, 21 Mar 2004, Matthew T. O'Connor wrote: > On Sun, 2004-03-21 at 20:31, Christopher Kings-Lynne wrote: > > > I think these configuration issues will become a lot easier if you make > > > the autovacuum daemon a subprocess of the postmaster (like, say, the > > > checkpoint process). Then yo

Re: [HACKERS] pg_autovacuum next steps

2004-03-21 Thread Matthew T. O'Connor
On Sun, 2004-03-21 at 18:12, Tom Lane wrote: > "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > > [ rtf document ] > > Please repost in some less proprietary format. Plain text is generally > considered the thing to use on this list. I don't think RTF is proprietary but I should have just pos

Re: [HACKERS] pg_autovacuum next steps

2004-03-21 Thread Matthew T. O'Connor
On Sun, 2004-03-21 at 20:31, Christopher Kings-Lynne wrote: > > I think these configuration issues will become a lot easier if you make > > the autovacuum daemon a subprocess of the postmaster (like, say, the > > checkpoint process). Then you have access to a host of methods for > > storing sta

Re: [HACKERS] pg_autovacuum next steps

2004-03-21 Thread Christopher Kings-Lynne
I think these configuration issues will become a lot easier if you make the autovacuum daemon a subprocess of the postmaster (like, say, the checkpoint process). Then you have access to a host of methods for storing state, handling configuration, etc. Yeah - why delay making it a backend proces

Re: [HACKERS] pg_autovacuum next steps

2004-03-21 Thread Peter Eisentraut
Matthew T. O'Connor wrote: > Lately I have been thinking about the next steps for the > pg_autovacuum daemon. I have written up a document that describes > what I'm planning to do next. Please read the attached and response > as I would really like some feedback. I think these configuration iss

  1   2   >