Re: [HACKERS] Partitioning/inherited tables vs FKs

2010-05-10 Thread Dmitry Fefelov
> The referential integrity triggers contain some extra magic that isn't > easily simulatable in userland, and that is necessary to make the > foreign key constraints airtight. We've discussed this previously but > I don't remember which thread it was or the details of when things > blow up. I th

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Fujii Masao
On Mon, May 10, 2010 at 3:27 PM, Simon Riggs wrote: > I already explained that killing the startup process first is a bad idea > for many reasons when shutdown was discussed. Can't remember who added > the new standby shutdown code recently, but it sounds like their design > was pretty poor if it

Re: [HACKERS] [BUGS] "SET search_path" clause ignored during function creation

2010-05-10 Thread Takahiro Itagaki
Tom Lane wrote: > Takahiro Itagaki writes: > > Thanks for the report! Please check whether the attached patch > > is the correct fix. An additional regression test is included. > > That's going to provoke "uninitialized variable" compiler warnings, > but otherwise it seems reasonably sane. I

Re: [HACKERS] Make archiver check for SIGHUP more often?

2010-05-10 Thread Fujii Masao
On Tue, May 11, 2010 at 10:13 AM, Tom Lane wrote: > There's a complaint over here > http://archives.postgresql.org/pgsql-general/2010-05/msg00365.php > about the archiver process not being very swift to adopt a new > value of archive_command.  This is because it only reacts to SIGHUP > once per ou

Re: [HACKERS] [GENERAL] Archiver not picking up changes to archive_command

2010-05-10 Thread Fujii Masao
On Tue, May 11, 2010 at 9:50 AM, Tom Lane wrote: > bricklen writes: >> Due to some heavy processing today, we have been falling behind on >> shipping log files (by about a 1000 logs or so), so wanted to up our >> bwlimit like so: > >> rsync -a %p postg...@192.168.80.174:/WAL_Archive/ && rsync >>

[HACKERS] Invitation to connect on LinkedIn

2010-05-10 Thread Carlos Jordão
LinkedIn Carlos Jordão souhaite se connecter à vous sur LinkedIn : -- Amine, I'd like to add you to my professional network on LinkedIn. - Carlos Accepter l'invitation de Carlos Jordão http://www.linkedin.com/e/VWCdriNHrcXMfYpwEG6pcCKmodzjIoHm

[HACKERS] Make archiver check for SIGHUP more often?

2010-05-10 Thread Tom Lane
There's a complaint over here http://archives.postgresql.org/pgsql-general/2010-05/msg00365.php about the archiver process not being very swift to adopt a new value of archive_command. This is because it only reacts to SIGHUP once per outer loop, ie, only after completing an archiving cycle. This

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Greg Smith
Josh Berkus wrote: The real question is how much of a speed-up fsync provides compared to the same workload with synchronous_commit disabled. The only case for fsync=off is one where that number is much faster. I can't say I've tested this. Most of my head-to-heads on fsync were before

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Josh Berkus
On 5/10/10 2:21 PM, Ross J. Reedstrom wrote: > On Mon, May 10, 2010 at 01:35:32PM -0700, Josh Berkus wrote: >> deleted, >> or on a reporting read-only clone of your database which gets >> recreated very >> night and is not used for failover. High quality hardware alone > > s/very/

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Josh Berkus
> The real question is how much of a speed-up fsync provides compared to > the same workload with synchronous_commit disabled. The only case for > fsync=off is one where that number is much faster. I can't say I've tested this. Most of my head-to-heads on fsync were before asych existed. --

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Greg Smith
Josh Berkus wrote: Wow. In a situation where you save seven minutes (4%), it's hardly worth turning off. I've had it be much higher, especially for really large databases. Cedric's system had a non-volatile write cache in it. In that case, a few percentage points of improvement is

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Florian Pflug
On May 10, 2010, at 17:39 , Kevin Grittner wrote: > Bruce Momjian wrote: >> Robert Haas wrote: > >>> Overall I would say opinion is about evenly split between: >>> >>> - leave it as-is >>> - make it a Boolean >>> - change it in some way but to something more expressive than a >>> Boolean > > I

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Ross J. Reedstrom
On Mon, May 10, 2010 at 01:35:32PM -0700, Josh Berkus wrote: > deleted, > or on a reporting read-only clone of your database which gets > recreated very > night and is not used for failover. High quality hardware alone s/very/every/ or s/very night/periodically/ Ross -- Ross Re

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Josh Berkus
> Wow. In a situation where you save seven minutes (4%), it's hardly > worth turning off. I've had it be much higher, especially for really large databases. -- -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Kevin Grittner
Cédric Villemain wrote: > On a recent pg_restore -j 32, with perc 6i with BBU, RAID10 8 hd, > results were not so bas with fsync turn on. (XFS with nobarrier su > and sw) > -- deactivate fsync > time pg_restore -U postgres -d foodb -j 32 foo.psql > real170m0.527s > user43m12.914s > sys

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Josh Berkus
> 1) Replace max_standby_delay with a boolean as per heikki's suggestion > > 2) Add an explicitly experimental option like max_standby_delay or > recovery_conflict_timeout which is only effective if you've chosen > recovery_conflict="pause recovery" > option and is explicitly documented as being

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Josh Berkus
All, Updated docs based on tracking this discussion. fsync through full page writes recorded below. fsync configuration parameter fsync (boolean) If this parameter is on, the PostgreSQL server will try to make sure th

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Cédric Villemain
2010/5/8 Bernd Helmle : > > > --On 7. Mai 2010 09:48:53 -0500 Kevin Grittner > wrote: > >> I think it goes beyond "tweaking" -- I think we should have a bald >> statement like "don't turn this off unless you're OK with losing the >> entire contents of the database cluster."  A brief listing of som

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Tom Lane
"Kevin Grittner" writes: > Perhaps Josh's language for fsync could be modified to work here > (we're now talking about full_page_writes, for anyone who's lost > track): > | it is only advisable to turn off fsync if you can easily recreate > | your entire database from external data. > That cove

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Kevin Grittner
"Joshua D. Drake" wrote: > The answer to this is: > > PostgreSQL.org recommends that this setting be left on at all > times. Turning it off, may lead to data corruption. > > Anything else is circumstantial and based on knowledge and facts > we don't have about environmental factors. Perhap

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Joshua D. Drake
On Mon, 2010-05-10 at 18:46 +0100, Greg Stark wrote: > On Mon, May 10, 2010 at 4:55 PM, Kevin Grittner > wrote: > > Robert Haas wrote: > > > >> "It might be safe" is a bit of a waffle. It would be nice if we > >> could provide some more clear guidance as to whether it is or is > >> not, or how s

Re: [HACKERS] [BUGS] "SET search_path" clause ignored during function creation

2010-05-10 Thread Tom Lane
Takahiro Itagaki writes: > Thanks for the report! Please check whether the attached patch > is the correct fix. An additional regression test is included. That's going to provoke "uninitialized variable" compiler warnings, but otherwise it seems reasonably sane. I don't particularly see the poi

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Greg Stark
On Mon, May 10, 2010 at 4:55 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> "It might be safe" is a bit of a waffle.  It would be nice if we >> could provide some more clear guidance as to whether it is or is >> not, or how someone could go about testing their hardware to find >> out. > > I

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Greg Stark
On Mon, May 10, 2010 at 5:20 PM, Bruce Momjian wrote: > You are right that we are much more flexible about changing > administrative configuration parameters (like this one) than SQL. In the > past, we even renamed logging parameters to be more consistent, and I > think that proves the bar is quit

Re: [HACKERS] make install fails due to "/bin/mkdir: missing operand"

2010-05-10 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Kenichiro Tanaka's message of lun may 10 07:07:27 -0400 2010: >> ./configure --enable-nls='UFT_JP' --prefix=/home/p900/posgrehome > I think this is pilot error, in the sense that it doesn't fail if you > don't pass an invalid language name. Maybe the bug is

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > Robert Haas wrote: > > >> Overall I would say opinion is about evenly split between: > >> > >> - leave it as-is > >> - make it a Boolean > >> - change it in some way but to something more expressive than a > >> Boolean > > I think a boolean w

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Stephen Frost
* Aidan Van Dyk (ai...@highrise.ca) wrote: > * Heikki Linnakangas [100510 06:03]: > > A problem with using the name "max_standby_delay" for Tom's suggestion > > is that it sounds like a hard limit, which it isn't. But if we name it > > something like: > > I'ld still rather an "if your killing som

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Kevin Grittner
Robert Haas wrote: > "It might be safe" is a bit of a waffle. It would be nice if we > could provide some more clear guidance as to whether it is or is > not, or how someone could go about testing their hardware to find > out. I think that the issue is that you could have corruption if some,

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Bruce Momjian
Robert Haas wrote: > On Mon, May 10, 2010 at 11:12 AM, Bruce Momjian wrote: > > Michael Tharp wrote: > >> On 05/08/2010 04:07 AM, Craig Ringer wrote: > >> > It's probably worth mentioning that people who want to turn off fsync to > >> > gain a performance boost should instead look at a RAID contro

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 11:12 AM, Bruce Momjian wrote: > Michael Tharp wrote: >> On 05/08/2010 04:07 AM, Craig Ringer wrote: >> > It's probably worth mentioning that people who want to turn off fsync to >> > gain a performance boost should instead look at a RAID controller with a >> > BBU so they

Re: [HACKERS] make install fails due to "/bin/mkdir: missing operand"

2010-05-10 Thread Alvaro Herrera
Excerpts from Kenichiro Tanaka's message of lun may 10 07:07:27 -0400 2010: > Reproduce case: > #prefix and with-pgport are not important > ./configure --enable-nls='UFT_JP' --prefix=/home/p900/posgrehome > --with-pgport=1900 > make && make install I think this is pilot error, in the sense that i

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Kevin Grittner
Bruce Momjian wrote: > Robert Haas wrote: >> Overall I would say opinion is about evenly split between: >> >> - leave it as-is >> - make it a Boolean >> - change it in some way but to something more expressive than a >> Boolean I think a boolean would limit the environments in which HS woul

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Mike Rylander
On Mon, May 10, 2010 at 6:03 AM, Heikki Linnakangas wrote: > Robert Haas wrote: >> On Thu, May 6, 2010 at 2:47 PM, Josh Berkus wrote: Now that I've realized what the real problem is with max_standby_delay (namely, that inactivity on the master can use up the delay), I think we shou

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Bruce Momjian
Michael Tharp wrote: > On 05/08/2010 04:07 AM, Craig Ringer wrote: > > It's probably worth mentioning that people who want to turn off fsync to > > gain a performance boost should instead look at a RAID controller with a > > BBU so they can safely enable write-back caching, getting most of the > >

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Robert Haas wrote: > Wultsch (who doesn't ever want to kill queries and therefore would be > happy with a boolean), Yeb Havinga (who never wants to stall recovery > and therefore would also be happy with a boolean), and Florian Pflug > (who points out that pause/resume is actually a nontrivial feat

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Simon Riggs wrote: > Bruce has used the word crippleware for the current state. Raising a > problem and then blocking solutions is the best way I know to cripple a > release. It should be clear that I've done my best to avoid this FYI, it was Robert Haas who used the term "crippleware" to describe

Re: [HACKERS] no universally correct setting for fsync

2010-05-10 Thread Michael Tharp
On 05/08/2010 04:07 AM, Craig Ringer wrote: It's probably worth mentioning that people who want to turn off fsync to gain a performance boost should instead look at a RAID controller with a BBU so they can safely enable write-back caching, getting most of the benefits of fsync=off safely. Which

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Andres Freund
On Monday 10 May 2010 14:00:45 Heikki Linnakangas wrote: > Florian Pflug wrote: > > On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: > >> If you're not going to apply any more WAL records before shutdown, you > >> could also just release all the AccessExclusiveLocks held by the startup > >> pr

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Florian Pflug wrote: > On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: >> If you're not going to apply any more WAL records before shutdown, you >> could also just release all the AccessExclusiveLocks held by the startup >> process. Whatever the transaction was doing with the locked relation,

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 6:13 AM, Florian Pflug wrote: > On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: >> If you're not going to apply any more WAL records before shutdown, you >> could also just release all the AccessExclusiveLocks held by the startup >> process. Whatever the transaction w

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 6:03 AM, Heikki Linnakangas wrote: > Yeah, I could live with that. > > A problem with using the name "max_standby_delay" for Tom's suggestion > is that it sounds like a hard limit, which it isn't. But if we name it > something like: > > # -1 = no timeout > # 0 = kill confli

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 2:27 AM, Simon Riggs wrote: > I already explained that killing the startup process first is a bad idea > for many reasons when shutdown was discussed. Can't remember who added > the new standby shutdown code recently, but it sounds like their design > was pretty poor if it

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Aidan Van Dyk
* Heikki Linnakangas [100510 06:03]: > A problem with using the name "max_standby_delay" for Tom's suggestion > is that it sounds like a hard limit, which it isn't. But if we name it > something like: I'ld still rather an "if your killing something, make sure you kill enough to get all the way

[HACKERS] make install fails due to "/bin/mkdir: missing operand"

2010-05-10 Thread Kenichiro Tanaka
Hello. This is my first mail to "pgsql-hackers@postgresql.org". When I install PostgreSQL,I get an error. So I'd like to send a report . PostgreSQL version: postgresql-9.0beta1 Operating system: CentOS release 5.4 (Final) Reproduce case: #prefix and with-pgport are not important ./configure --en

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Florian Pflug
On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: > If you're not going to apply any more WAL records before shutdown, you > could also just release all the AccessExclusiveLocks held by the startup > process. Whatever the transaction was doing with the locked relation, if > we're not going to r

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Robert Haas wrote: > On Thu, May 6, 2010 at 2:47 PM, Josh Berkus wrote: >>> Now that I've realized what the real problem is with max_standby_delay >>> (namely, that inactivity on the master can use up the delay), I think >>> we should do what Tom originally suggested here. It's not as good as >>>

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Robert Haas wrote: > On Sun, May 9, 2010 at 6:58 PM, Andres Freund wrote: >> On Monday 10 May 2010 00:25:44 Florian Pflug wrote: >>> On May 9, 2010, at 22:01 , Robert Haas wrote: On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine >> wrote: Seems like it could take FOREVER on a busy system

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Dimitri Fontaine
Robert Haas writes: > On Sun, May 9, 2010 at 6:58 PM, Andres Freund wrote: >> The difference is that in HS you have to wait for a moment where *no >> exclusive >> lock at all* exist, possibly without contending for any of them, while on the >> master you might not even blocked by the existence o