Re: [HACKERS] Simplifying replication

2010-10-27 Thread Josh Berkus
I sort of agree with you that the current checkpoint_segments parameter is a bit hard to tune, at least if your goal is to control the amount of disk space that will be used by WAL files. But I'm not sure your proposal is better. Instead of having a complicated formula for predicting how much

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Andrew Dunstan
On 10/27/2010 11:38 PM, Tom Lane wrote: Andrew Dunstan writes: But I think we can do better than this. We should really pass an hashref with the record's column names as keys rather than just calling record_out. I'll work on that. Definitely. If you aren't providing that info then it's hard

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Tom Lane
Andrew Dunstan writes: > But I think we can do better than this. We should really pass an hashref > with the record's column names as keys rather than just calling > record_out. I'll work on that. Definitely. If you aren't providing that info then it's hard to write a generic function, which i

Re: [HACKERS] An unfortunate logging behavior when (mis)configuring recovery.conf

2010-10-27 Thread Daniel Farina
On Wed, Oct 27, 2010 at 6:44 PM, Jaime Casanova wrote: > On Wed, Oct 27, 2010 at 6:18 PM, Daniel Farina wrote: >> >> As a result, "aborting startup due to startup process failure" is seen >> in the log, but not the messages seen in >> xlog.c:readRecoveryCommandFile that triggered the failure. >>

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Andrew Dunstan
On 10/25/2010 09:32 PM, Andrew Dunstan wrote: On 10/25/2010 07:12 PM, Tom Lane wrote: However, that objection doesn't hold for plperl or pltcl (and likely not plpython, though I don't know that language enough to be sure). So it would be a reasonable feature request to teach those PLs to acce

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Andrew Dunstan
On 10/25/2010 09:32 PM, Andrew Dunstan wrote: On 10/25/2010 07:12 PM, Tom Lane wrote: However, that objection doesn't hold for plperl or pltcl (and likely not plpython, though I don't know that language enough to be sure). So it would be a reasonable feature request to teach those PLs to acce

Re: [HACKERS] An unfortunate logging behavior when (mis)configuring recovery.conf

2010-10-27 Thread Jaime Casanova
On Wed, Oct 27, 2010 at 6:18 PM, Daniel Farina wrote: > > As a result, "aborting startup due to startup process failure" is seen > in the log, but not the messages seen in > xlog.c:readRecoveryCommandFile that triggered the failure. > can you explain what steps you did to reproduce this? -- Jai

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Robert Haas
On Wed, Oct 27, 2010 at 5:01 PM, Josh Berkus wrote: > >> You have to put the WAL files *somewhere* while you do the base backup. >> PostgreSQL can't itself work out where that is, nor can it work out >> ahead of time how big it will need to be, since it is up to you how you >> do your base backup.

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Robert Haas
On Wed, Oct 27, 2010 at 3:53 AM, Simon Riggs wrote: > On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote: >> On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus wrote: >> > >> >> If you set wal_keep_segments=0, archive_mode=on, and >> >> archive_command=, you might run out of disk space. >> >> >> >> I

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
> None of us know. What I do know is that I don't want PostgreSQL to be > slower out of the box. Understandable. So it seems like the answer is getting replication down to one configuration variable for the common case. That eliminates the cycle of "oops, need to set X and restart/reload" with

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Joshua D. Drake
On Wed, 2010-10-27 at 19:52 -0400, Robert Haas wrote: > On Wed, Oct 27, 2010 at 7:45 PM, Josh Berkus wrote: > >> I would also agree that the minority of our users will want replication. > >> The majority of CMD customers, PGX customers, EDB Customers will want > >> replication but that is by far N

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Robert Haas
On Wed, Oct 27, 2010 at 7:45 PM, Josh Berkus wrote: >> I would also agree that the minority of our users will want replication. >> The majority of CMD customers, PGX customers, EDB Customers will want >> replication but that is by far NOT the majority of our (.Org) users. > > That just means that

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
> I would also agree that the minority of our users will want replication. > The majority of CMD customers, PGX customers, EDB Customers will want > replication but that is by far NOT the majority of our (.Org) users. That just means that *you don't know* how many .org users plan to implement rep

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Joshua D. Drake
On Wed, 2010-10-27 at 16:13 -0700, Josh Berkus wrote: > > That's not even considering the extra WAL that is generated when you > > move up from wal_level = "minimal". That's probably the bigger > > performance issue in practice. > > Yeah, I think we've established that we can't change that. > >

[HACKERS] An unfortunate logging behavior when (mis)configuring recovery.conf

2010-10-27 Thread Daniel Farina
Hello list, I just encountered an interesting undesirable behavior in Postgres 9.0's error reporting dealing with (trivially) malformed recovery.conf, as might be the case when setting up hot standby. In this case, there were some missing fields, and they were checked as they are supposed to be in

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
> That's not even considering the extra WAL that is generated when you > move up from wal_level = "minimal". That's probably the bigger > performance issue in practice. Yeah, I think we've established that we can't change that. > I said, and meant, that you didn't make the case at all; you just

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Tom Lane
Josh Berkus writes: >> You're assuming that we should set up the default behavior to support >> replication and penalize those who aren't using it. > What's the penalty? Simon just said that there isn't one. I don't know what Simon is thinking, but I think he's nuts. There is is obvious extra

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Greg Stark
On Wed, Oct 27, 2010 at 12:33 PM, Tom Lane wrote: > You're assuming that we should set up the default behavior to support > replication and penalize those who aren't using it.  Considering that > we haven't even *had* replication until now, it seems a pretty safe > bet that the majority of our use

Re: [HACKERS] crash in plancache with subtransactions

2010-10-27 Thread Tom Lane
I wrote: >> Heikki Linnakangas writes: >>> One simple idea is to keep a flag along with the executor state to >>> indicate that the executor state is currently in use. Set it just before >>> calling ExecEvalExpr, and reset afterwards. If the flag is already set >>> in the beginning of exec_eval

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Josh Berkus
> You have to put the WAL files *somewhere* while you do the base backup. > PostgreSQL can't itself work out where that is, nor can it work out > ahead of time how big it will need to be, since it is up to you how you > do your base backup. Setting a parameter to -1 doesn't make the problem > go a

Re: [HACKERS] foreign keys for array/period contains relationships

2010-10-27 Thread Josh Berkus
On 10/26/10 11:53 AM, Jeff Davis wrote: > Intuitively, I would expect all 1's to be replaced by 6's in all arrays. > But I can now see why you would be hesitant to try to support that. If people want cascading update to work, they shouldn't be denormalizing. --

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
> You're assuming that we should set up the default behavior to support > replication and penalize those who aren't using it. What's the penalty? Simon just said that there isn't one. And there's a difference between saying that I "failed to make a case" vs. "the cost is too great". Saying the

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Fujii Masao
On Wed, Oct 27, 2010 at 3:03 AM, fazool mein wrote: > >> Might I suggest adopting the same technique walsender does, ie just read >> the data back from disk?  There's a reason why we gave up trying to have >> walsender read directly from the buffers. >> > > That is exactly what I do not want to do

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Joshua D. Drake
On Wed, 2010-10-27 at 15:33 -0400, Tom Lane wrote: > Josh Berkus writes: > >>> Josh has completely failed to make a case that > >>> that should be the default. > >> > >> Agreed. > > > In what way have a failed to make a case? > > You're assuming that we should set up the default behavior to sup

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Tom Lane
"Kevin Grittner" writes: > Josh Berkus wrote: >>> Except that changing wal_keep_segments doesn't require restarting >>> the master. >> >> Our docs say that it does: >> This parameter can only be set in the postgresql.conf file or on >> the server command line. > That sounds as though a reload

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Tom Lane
Josh Berkus writes: >>> Josh has completely failed to make a case that >>> that should be the default. >> >> Agreed. > In what way have a failed to make a case? You're assuming that we should set up the default behavior to support replication and penalize those who aren't using it. Considering

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Simon Riggs
On Wed, 2010-10-27 at 10:05 -0700, Josh Berkus wrote: > >> Josh has completely failed to make a case that > >> that should be the default. > > > > Agreed. > > In what way have a failed to make a case? I just removed a huge hurdle on the journey to simplification. That doesn't mean I think you hav

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-10-27 Thread Heikki Linnakangas
On 26.10.2010 18:34, Boszormenyi Zoltan wrote: thank you very much for pointing me to dynahash, here is the next version that finally seems to work. Two patches are attached, the first is the absolute minimum for making it work, this still has the Tree type for canon_pathkeys and eq_classes got

[HACKERS] Bikeshedding on enum vocabulary

2010-10-27 Thread Aidan Van Dyk
On Wed, Oct 27, 2010 at 10:57 AM, Alvaro Herrera wrote: > Wow, this must be the most difficult smallest thing I have ever seen > discussed in pg-hackers.  It doesn't seem like there are enough votes > in any particular direction.  Now *this* is proper bikeshedding. > > Should we ask more openly i

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Kevin Grittner
Josh Berkus wrote: >> Except that changing wal_keep_segments doesn't require restarting >> the master. > > Our docs say that it does: > This parameter can only be set in the postgresql.conf file or on > the server command line. That sounds as though a reload would do it; I don't see that indi

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Josh Berkus
It is the same to the user either way. In either case you have to change some settings and restart the master. Except that changing wal_keep_segments doesn't require restarting the master. Our docs say that it does: This parameter can only be set in the postgresql.conf file or on the serve

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Alvaro Herrera
Excerpts from Markus Wanner's message of mié oct 27 11:44:20 -0300 2010: > On 10/26/2010 05:52 PM, Alvaro Herrera wrote: > > And horrible for performance, I imagine. Those locks are highly trafficked. > > Note, however, that offloading this to the file-system just moves > congestion there. So we

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Alvaro Herrera
Excerpts from Andrew Dunstan's message of mié oct 27 11:18:44 -0300 2010: > The reason I chose LABEL was that it's consistent with what we have used > elsewhere, both in the docs and the catalog. But I'm not strongly wedded > to it. If it's a choice between ELEMENT and VALUE, I too prefer VALUE

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Andrew Dunstan
On 10/27/2010 10:00 AM, Kevin Grittner wrote: Alvaro Herrera wrote: Excerpts from Dean Rasheed's message: Well ELEMENT is a reserved keyword in SQL:2008, to support multisets, so if we ever supported that feature... Hah! Well, here's a patch for LABEL in any case. If we're going to have

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Markus Wanner
On 10/26/2010 05:52 PM, Alvaro Herrera wrote: > And horrible for performance, I imagine. Those locks are highly trafficked. Note, however, that offloading this to the file-system just moves congestion there. So we are effectively saying that we expect filesystems to do a better job (in that aspec

[HACKERS] Tracking latest timeline in standby mode

2010-10-27 Thread Heikki Linnakangas
At the moment, when you specify recovery_target_timeline='latest', we scan for the latest timeline at the beginning of recovery, and pick that as the target. If new timelines appear during recovery, we stick to the target chosen in the beginning, the new timelines are ignored. That's undesirabl

Re: [HACKERS] Re: ECPG dynamic cursor fix for UPDATE/DELETE ... WHERE CURRENT OF :curname

2010-10-27 Thread Boszormenyi Zoltan
Hi, Michael Meskes írta: >> 1. The statement >> >> UPDATE table SET fld1 = :input1 >> WHERE CURRENT OF :curname >> RETURNING id + :input2; >> >> is transformed into >> >> UPDATE table SET fld1 = $1 >> WHERE CURRENT OF $0 >> RETURNING id + $2; >> >> and the $0 is pas

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Andrew Dunstan
On 10/26/2010 09:16 PM, Tom Lane wrote: But ... having said all that, I have to agree that ELEMENT seems preferable to LABEL if we ignore micro-considerations of parser efficiency --- I still think LABEL is a pretty poor choice of word here. Personally I'd still take VALUE as being my first

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Kevin Grittner
Alvaro Herrera wrote: > Excerpts from Dean Rasheed's message: >> Well ELEMENT is a reserved keyword in SQL:2008, to support >> multisets, so if we ever supported that feature... > > Hah! > > Well, here's a patch for LABEL in any case. If we're going to > have to reserve ELEMENT in the future

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Simon Riggs
On Tue, 2010-10-19 at 15:32 -0400, Tom Lane wrote: > Robert Haas writes: > > On Tue, Oct 19, 2010 at 12:18 PM, Josh Berkus wrote: > >> On 10/19/2010 09:06 AM, Greg Smith wrote: > >>> I think Magnus's idea to bump the default to 5 triages the worst of the > >>> annoyance here, without dropping the

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Dean Rasheed
On 27 October 2010 02:16, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Dean Rasheed's message of mar oct 26 15:46:56 -0300 2010: >>> Well ELEMENT is a reserved keyword in SQL:2008, to support multisets, >>> so if we ever supported that feature... > >> Hah! > > Hmmm ... I dug through

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Simon Riggs
On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote: > On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus wrote: > > > >> If you set wal_keep_segments=0, archive_mode=on, and > >> archive_command=, you might run out of disk space. > >> > >> If you set wal_keep_segments=-1, you might run out of disk spa