Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Fri, 2009-02-06 at 00:07 +, Greg Stark wrote: > On Thu, Feb 5, 2009 at 11:57 PM, Simon Riggs wrote: > > > > Writing an application maintenance utility in PL/pgSQL is much better > > than having to write it for all the different servers an application may > > need to run on. > > Welcome to

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 19:23 -0500, Andrew Dunstan wrote: > > Simon Riggs wrote: > >> not trying to make sure that cron-driven > >> scripts can do anything autovacuum can. > >> > > > > I'm not in favour of limiting our capability to internal actions only. > > If we add a capability for schedu

[HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1522)

2009-02-05 Thread KaiGai Kohei
The following patches are updated ones: [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1522.patch [2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1522.patch [3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1522.patch

Re: [HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Robert Haas
> I think the only fix for this that's practical in the 8.4 time frame is > to give up trying to flatten EXISTS/NOT EXISTS that occur within the ON > condition of an outer join, ie, revert to 8.3's level of intelligence > about this case. It seems like a general purpose solution would involve > so

Re: [HACKERS] 8.4 release planning

2009-02-05 Thread Koichi Suzuki
I understand Simon is extremely busy on his own patch. I appreciate if anyone help the review. 2009/2/6 Fujii Masao : > Hi, > > On Tue, Feb 3, 2009 at 5:08 AM, Bruce Momjian wrote: >>> o Others >> >> We will focus on all the other items on the commit fest page, and that >> will determine

Re: [HACKERS] 8.4 release planning

2009-02-05 Thread Fujii Masao
Hi, On Tue, Feb 3, 2009 at 5:08 AM, Bruce Momjian wrote: >> o Others > > We will focus on all the other items on the commit fest page, and that > will determine our time-line for 8.4 beta, i.e. the first three items > will not delay our beta release. Simon is assigned as reviewer of PITR

Re: [HACKERS] Synch Replication

2009-02-05 Thread Fujii Masao
Hi Niranjan, On Thu, Feb 5, 2009 at 10:50 PM, K, Niranjan (NSN - IN/Bangalore) wrote: >> I changed the code to dump all messages for replication, so >> please try it and report the result. The messages which the >> standby server receives are logged in the file (*1). Please >> send also that file

Re: [HACKERS] autovacuum and reloptions

2009-02-05 Thread Euler Taveira de Oliveira
Alvaro Herrera escreveu: > So here's what looks like a committable patch. > > Note to self: remember to remove src/include/catalog/pg_autovacuum.h and > to bump catversion. > > Works for me. Just a few comments. (i) I don't like this construction "by entries by changing storage parameters". I p

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Robert Haas
On Thu, Feb 5, 2009 at 7:13 PM, Robert Haas wrote: > Thinking about this a little more, the biggest problem I have with > this feature is that it makes autovacuum_enabled mean two different > things depending on context. But maybe we should change the name of > the reloption to "autovacuum" and h

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Josh Berkus
Alvaro, First off, with over 200 GUC variables currently active, in general we should be looking to *eliminate* variables rather than adding them. So my personal bar for endorsing a new GUC is set pretty high. Now, sometimes it might make more sense to keep it enabled but have it only check

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Andrew Dunstan
Simon Riggs wrote: not trying to make sure that cron-driven scripts can do anything autovacuum can. I'm not in favour of limiting our capability to internal actions only. If we add a capability for scheduling work, we can easily make it capable of scheduling many kinds of work. Writing

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Robert Haas
>> Agreed, let's get this capability out in 8.4 and we can always adust it >> based on user demand. > > Oh, I agree to limiting what we do for 8,4, but we need more later. Thinking about this a little more, the biggest problem I have with this feature is that it makes autovacuum_enabled mean two d

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Greg Stark
On Thu, Feb 5, 2009 at 11:57 PM, Simon Riggs wrote: > > Writing an application maintenance utility in PL/pgSQL is much better > than having to write it for all the different servers an application may > need to run on. Welcome to the suction effect. If your scheduler is in the database then you'r

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 17:08 -0500, Tom Lane wrote: > Simon Riggs writes: > > On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: > >> It'd make more sense to put the effort into developing > >> better scheduling control over autovacuum, such as a concept of > >> maintenance windows. > > > We need

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 17:57 -0500, Bruce Momjian wrote: > Tom Lane wrote: > > Simon Riggs writes: > > > On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: > > >> It'd make more sense to put the effort into developing > > >> better scheduling control over autovacuum, such as a concept of > > >> ma

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs writes: > > On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: > >> It'd make more sense to put the effort into developing > >> better scheduling control over autovacuum, such as a concept of > >> maintenance windows. > > > We need that as well, not instead of. > >

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Joshua D. Drake
On Thu, 2009-02-05 at 17:08 -0500, Tom Lane wrote: > I disagree; adding every frammish anyone could ever think of is not > an overall improvement to the system. > :) > My feeling is that we should be trying to eliminate use-cases for > cron-driven vacuuming, not trying to make sure that cron-dr

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 18:54 -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > > > On Thu, 2009-02-05 at 18:25 -0300, Alvaro Herrera wrote: > > > > > So you're not aware that we're doing away with pg_autovacuum for good? > > > It's going to be replaced by reloptions, i.e. > > > ALTER TABLE fo

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Tom Lane
Simon Riggs writes: > On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: >> It'd make more sense to put the effort into developing >> better scheduling control over autovacuum, such as a concept of >> maintenance windows. > We need that as well, not instead of. I disagree; adding every frammish

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Simon Riggs wrote: > > On Thu, 2009-02-05 at 18:25 -0300, Alvaro Herrera wrote: > > > So you're not aware that we're doing away with pg_autovacuum for good? > > It's going to be replaced by reloptions, i.e. > > ALTER TABLE foo SET (autovacuum_enabled = false); > > > > Obviously there's no way to

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Tom Lane wrote: > (BTW, autovac does vacuum tables to prevent wraparound even if you try > to tell it to skip them, right?) Yes. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Euler Taveira de Oliveira
Alvaro Herrera escreveu: > Hi, > > Right now, when autovacuum is turned on we always assume it's supposed > to process all tables except those that have autovacuum_enabled=false. > > Now, sometimes it might make more sense to keep it enabled but have it > only check for certain tables, and leave

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 18:25 -0300, Alvaro Herrera wrote: > So you're not aware that we're doing away with pg_autovacuum for good? > It's going to be replaced by reloptions, i.e. > ALTER TABLE foo SET (autovacuum_enabled = false); > > Obviously there's no way to add a "catchall" setting. Seems l

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: > It'd make more sense to put the effort into developing > better scheduling control over autovacuum, such as a concept of > maintenance windows. We need that as well, not instead of. People want to be able to specify (as an example) * autovac

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Simon Riggs wrote: > > On Thu, 2009-02-05 at 17:45 -0300, Alvaro Herrera wrote: > > > For this we'd have a separate GUC parameter, as in $SUBJECT (I'm not > > wedded to the name), and have the user set autovacuum_enabled=true via > > reloptions to enable it. > > I would prefer it if that behavio

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Robert Haas
On Thu, Feb 5, 2009 at 3:45 PM, Alvaro Herrera wrote: > Right now, when autovacuum is turned on we always assume it's supposed > to process all tables except those that have autovacuum_enabled=false. > > Now, sometimes it might make more sense to keep it enabled but have it > only check for certai

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Tom Lane
Alvaro Herrera writes: > Right now, when autovacuum is turned on we always assume it's supposed > to process all tables except those that have autovacuum_enabled=false. > Now, sometimes it might make more sense to keep it enabled but have it > only check for certain tables, and leave the majority

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 17:45 -0300, Alvaro Herrera wrote: > Right now, when autovacuum is turned on we always assume it's supposed > to process all tables except those that have autovacuum_enabled=false. > > Now, sometimes it might make more sense to keep it enabled but have it > only check for c

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Joshua D. Drake
On Thu, 2009-02-05 at 17:45 -0300, Alvaro Herrera wrote: > Hi, > > Right now, when autovacuum is turned on we always assume it's supposed > to process all tables except those that have autovacuum_enabled=false. > > Now, sometimes it might make more sense to keep it enabled but have it > only chec

[HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Hi, Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it only check for certain tables, and leave the majority of them disabled. For this

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 21:54 +0200, Heikki Linnakangas wrote: > - If bgwriter is performing a restartpoint when recovery ends, the > startup checkpoint will be queued up behind the restartpoint. And since > it uses the same smoothing logic as checkpoints, it can take quite some > time for that

Re: [HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> If it referred to any variables of the left side of the upper join >> (ie, "C" here) then we couldn't convert it to a separate join at > all. > The only thing that comes to mind for me that seems possibly helpful > is that we have typically conside

Re: [HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Kevin Grittner
>>> Tom Lane wrote: > SELECT ... FROM > "Case" "C" > LEFT OUTER JOIN "CaseDispo" "CD" > ON ("CD"."caseNo" = "C"."caseNo") AND ("CD"."countyNo" = "C"."countyNo") > AND (NOT (EXISTS (SELECT 1 FROM "CaseDispo" "CD2" > WHERE ("CD2"."caseNo" = "CD"."caseNo") >

[HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Tom Lane
I looked into the planning problems in HEAD discussed here: http://archives.postgresql.org/pgsql-hackers/2009-02/msg00120.php As I suspected, there are really two independent bugs there, though both are in the new semi/antijoin planning code. The first problem is the one already mentioned: cost_me

[HACKERS] Re: [BUGS] BUG #4516: FOUND variable does not work after RETURN QUERY

2009-02-05 Thread Bruce Momjian
Pavel Stehule wrote: > I am sending patch, that adds FOUND and GET DIAGNOSTICS support for > RETURN QUERY statement Updated patch attached and applied. Thanks. --- > > Regards > Pavel Stehule > > > > 2008/11/10 Andrew

Re: [HACKERS] Auto-updated fields

2009-02-05 Thread Robert Haas
> At the very least it should not have a hard-coded field name in it. You > should pass the field name to be set as a parameter in the trigger setup. > > That's probably a lot more doable if the trigger is written in C, and in any > case I think any prepackaged triggers we provide should be written

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 14:18 +0200, Heikki Linnakangas wrote: > when the control file is updated in XLogFlush, it's > typically the bgwriter doing it as it cleans buffers ahead of the > clock hand, not the startup process That is the key point. Let's do it your way. -- Simon Riggs w

Re: [HACKERS] Auto-updated fields

2009-02-05 Thread Andrew Dunstan
Christopher Browne wrote: I think this is a TODO, but not sure who is working on it or what needs to be done. The generic version in pagila is perhaps not generic enough: CREATE FUNCTION last_updated() RETURNS trigger AS $$ BEGIN NEW.last_update = CURRENT_TIMESTAMP; RETURN NEW; EN

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 13:18 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: So we might end up flushing more often *and* we will be doing it potentially in the code path of other users. For

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 13:18 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: > >> Simon Riggs wrote: > > > >>> So we might end up flushing more often *and* we will be doing it > >>> potentially in the code path of other users.

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: So we might end up flushing more often *and* we will be doing it potentially in the code path of other users. For example, imagine a database that fits completely in shared buffers. If we updat

Re: [HACKERS] Synch Replication

2009-02-05 Thread Fujii Masao
Hi Niranjan, On Thu, Feb 5, 2009 at 3:34 PM, K, Niranjan (NSN - IN/Bangalore) wrote: >>I added the further logging codes >> into the patch, and uploaded the source. If you have time, >> please try it and report the results. > > Refer attached logs. Thanks, but I have not identified the cause yet

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > So we might end up flushing more often *and* we will be doing it > > potentially in the code path of other users. > > For example, imagine a database that fits completely in shared buffers. > If we update at e

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 10:31 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: I got rid of minSafeStartPoint, advancing minRecoveryPoint instead. And it's advanced in XLogFlush instead of XLogFileRead. I'll post

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 09:26 +, Simon Riggs wrote: > This change seems speculative and also against what has previously been > agreed with Tom. If he chooses not to comment on your changes, that's up > to him, but I don't think you should remove things quietly that have > been put there throug

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 10:31 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: > > > I've changed the way minRecoveryPoint is updated now anyway, so it no > longer happens every XLogFileRead(). > >>> Care to elucidate?

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 10:07 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: > >> Simon Riggs wrote: > >>> I would suggest that at end of recovery we write the last LSN to the > >>> control file, so if we crash recover then we w

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: I've changed the way minRecoveryPoint is updated now anyway, so it no longer happens every XLogFileRead(). Care to elucidate? I got rid of minSafeStartPoint, advancing minRecoveryPoint instead. And it's advanced i

Re: [HACKERS] Auto-updated fields

2009-02-05 Thread Stephen R. van den Berg
Christopher Browne wrote: >On Wed, Feb 4, 2009 at 1:23 PM, Bruce Momjian wrote: >> Robert Treat wrote: >>> On Wednesday 21 January 2009 20:21:41 Bruce Momjian wrote: >>> CREATE FUNCTION last_updated() RETURNS trigger >>> AS $$ >>> BEGIN >>> NEW.last_update = CURRENT_TIMESTAMP; >>> RETU

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: I would suggest that at end of recovery we write the last LSN to the control file, so if we crash recover then we will always end archive recovery at the same place again should we re-enter it. So