Re: [HACKERS] WIP: Deferrable unique constraints

2009-07-08 Thread Dean Rasheed
Jeff Davis wrote: > First, I'm happy that you're working on this; I think it's important. I > am working on another index constraints feature that may have some > interaction: > > http://archives.postgresql.org/pgsql-hackers/2009-07/msg00302.php > > Let me know if you see any potential conflicts

Re: [HACKERS] WIP: generalized index constraints

2009-07-08 Thread Dean Rasheed
Tom Lane wrote: > ... I think it might be interesting to turn > around Jeff's syntax sketch and provide a way to say that a CONSTRAINT > declaration should depend on some previously added index, eg > something like > > ALTER TABLE tab ADD CONSTRAINT UNIQUE (col1, col2) USING index > Is th

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Dimitri Fontaine
Hi, Tom Lane writes: > I think this fails the basic sanity check: do you need it to still work > when the master is dead. I don't get it. Why would we want to setup a slave against a dead master? The way I understand the current design of Synch Rep, when you start a new slave the following ha

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Jan Urbański
Tom Lane wrote: > "Kevin Grittner" writes: >> I guess the question is whether there is anyone who has had a contrary >> experience. (There must have been some benchmarks to justify adding >> geqo at some point?) > > The CVS history shows that geqo was integrated on 1997-02-19, which > I think me

Re: [HACKERS] bytea vs. pg_dump

2009-07-08 Thread Bernd Helmle
--On Dienstag, Juli 07, 2009 18:07:08 -0400 Tom Lane wrote: Enum. If we do this then it seems entirely fair that someone might want other settings someday. Also, it seems silly to pick a format partly on the grounds that it's expansible, and then not make the control GUC expansible. Perhaps

Re: [HACKERS] WIP: Deferrable unique constraints

2009-07-08 Thread Dean Rasheed
Jeff Davis wrote: > On Tue, 2009-07-07 at 19:38 +0100, Dean Rasheed wrote: >> This approach works well if the number of potential conflicts is >> small. > > [...] > >> Curing the scalability problem by spooling the queue to disk shouldn't >> be too hard to do, but that doesn't address the problem

Re: [HACKERS] bytea vs. pg_dump

2009-07-08 Thread Pavel Stehule
2009/7/8 Bernd Helmle : > --On Dienstag, Juli 07, 2009 18:07:08 -0400 Tom Lane > wrote: > >> Enum.  If we do this then it seems entirely fair that someone might >> want other settings someday.  Also, it seems silly to pick a format >> partly on the grounds that it's expansible, and then not make t

Re: [HACKERS] pgxs and make check message

2009-07-08 Thread Peter Eisentraut
On Monday 06 July 2009 22:42:54 Peter Eisentraut wrote: > Isn't it a bad idea that this from pgxs.mk does not return a non-zero > status? > > check: > @echo "'make check' is not supported." > @echo "Do 'make install', then 'make installcheck' instead." > > Or is something relying on a nonex

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Kevin Grittner
Dimitri Fontaine wrote: > 4. sync: slave is no more lagging, it's applying the stream as it > gets it, either as part of the master transaction or not > depending on the GUC settings I think the interesting bit is when you're at this point and the connection between the master and sl

Re: [HACKERS] pg_migrator mention in documentation

2009-07-08 Thread Bruce Momjian
Peter Eisentraut wrote: > On Friday 03 July 2009 02:28:22 Bruce Momjian wrote: > > I looked at that and the problem is that pg_migrator must be built > > against the _new_ source tree, and will issue an error and exit if it > > isn't. The problem with PGXS is it silently chooses the source tree to

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Dimitri Fontaine
"Kevin Grittner" writes: > Dimitri Fontaine wrote: > >> 4. sync: slave is no more lagging, it's applying the stream as it >> gets it, either as part of the master transaction or not >> depending on the GUC settings > > I think the interesting bit is when you're at this point and th

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Mark Mielke
On 07/08/2009 09:59 AM, Kevin Grittner wrote: Dimitri Fontaine wrote: 4. sync: slave is no more lagging, it's applying the stream as it gets it, either as part of the master transaction or not depending on the GUC settings I think the interesting bit is when you're at t

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Heikki Linnakangas
Mark Mielke wrote: > On 07/08/2009 09:59 AM, Kevin Grittner wrote: >> I think the interesting bit is when you're at this point and the >> connection between the master and slave goes down for a couple days. >> How do you handle that? > > Been following with great interest... > > If the updates ar

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Noah Misch
On Tue, Jul 07, 2009 at 09:31:14AM -0500, Kevin Grittner wrote: > I don't remember any clear resolution to the wild variations in plan > time mentioned here: > > http://archives.postgresql.org/pgsql-hackers/2009-06/msg00743.php > > I think it would be prudent to try to figure out why small chan

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from theprimary to the standby

2009-07-08 Thread Kevin Grittner
Heikki Linnakangas wrote: > But more importantly, it can happen by accident. Someone trips on > the power plug of the slave on Friday, and it goes unnoticed until > Monday when DBA comes to work. We've had people unplug things by accident exactly that way. :-/ We've also had replication ac

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Alvaro Herrera
Itagaki Takahiro wrote: > Is it acceptable to use pthread in contrib module? We don't have a precedent it seems. I think the requirement would be that it should compile if pthread support is not present. > If ok, I will add the patch to the next commitfest. Add it anyway -- discussion should h

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Heikki Linnakangas
Fujii Masao wrote: > On Wed, Jul 8, 2009 at 4:00 AM, Heikki > Linnakangas wrote: >>> I would envision the slaves >>> connecting to the master's replication port and asking "feed me WAL >>> beginning at LSN position thus-and-so", with no notion of WAL file >>> boundaries exposed anyplace. >> Yep, t

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Heikki Linnakangas
Alvaro Herrera wrote: > Itagaki Takahiro wrote: > >> Is it acceptable to use pthread in contrib module? > > We don't have a precedent it seems. I think the requirement would be > that it should compile if pthread support is not present. My thoughts as well. But I wonder, would it be harder or e

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Tom Lane
Alvaro Herrera writes: > Itagaki Takahiro wrote: >> Is it acceptable to use pthread in contrib module? > We don't have a precedent it seems. I think the requirement would be > that it should compile if pthread support is not present. Right. Breaking it for non-pthread environments is not accep

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Andrew Dunstan
Heikki Linnakangas wrote: Alvaro Herrera wrote: Itagaki Takahiro wrote: Is it acceptable to use pthread in contrib module? We don't have a precedent it seems. I think the requirement would be that it should compile if pthread support is not present. My thoughts as wel

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Stefan Kaltenbrunner
Tom Lane wrote: Alvaro Herrera writes: Itagaki Takahiro wrote: Is it acceptable to use pthread in contrib module? We don't have a precedent it seems. I think the requirement would be that it should compile if pthread support is not present. Right. Breaking it for non-pthread environment

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Greg Smith
On Wed, 8 Jul 2009, Itagaki Takahiro wrote: Multi-threading would be a solution. The attached patch adds -j (number of jobs) option to pgbench. Should probably name this -w "numbers of workers" to stay consistent with terminology used on the server side. Is it acceptable to use pthread in

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Kenneth Marshall
Hi, When I was first familiarizing myself with PostgreSQL, I took a walk through its documentation on GECO and similar processes in the literature. One big advantage of GECO is that you can trade off planning time for plan optimization. I do agree that it should be updated, but there were definite

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Tom Lane
Andrew Dunstan writes: > I think you should have it use pthreads if available, or Windows threads > there, or fork() elsewhere. Hmm, but how will you communicate stats back from the sub-processes? pg_restore doesn't need anything more than a success/failure result from its child processes, but I

[HACKERS] 8.4.0 vs. locales vs. pl/perl?

2009-07-08 Thread Andrew Gierth
Having been helping someone out on IRC with what looked initially like an index corruption problem, it turns out to be (almost certainly) a locale initialization error. environment: debian packaged 8.4.0 scenario: restoring a dump results in incorrect indexes for some specific tables (indexscan o

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan writes: I think you should have it use pthreads if available, or Windows threads there, or fork() elsewhere. Hmm, but how will you communicate stats back from the sub-processes? pg_restore doesn't need anything more than a success/failure result from i

Re: [HACKERS] New types for transparent encryption

2009-07-08 Thread Chris Browne
gsst...@mit.edu (Greg Stark) writes: > However I have a different concern which hasn't been raised yet. > Encrypting lots of small chunks of data with the same key is a very > dangerous thing to do and it's very tricky to get right. Yeah, that's exactly the sort of thing that would be Most Useful

Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-08 Thread Chris Browne
robertmh...@gmail.com (Robert Haas) writes: > I suspect both are true, but in the unlikely event that we decide on > some massive change to the system, we can either run the DBs in > parallel as Tom suggests, or dump out the older data in Wiki markup > and post it on there. But I can't imagine what

Re: [HACKERS] New types for transparent encryption

2009-07-08 Thread Chris Browne
a...@esilo.com (Andrew Chernow) writes: > Would the IV be regenerated every time the plaintext is updated, to > avoid using it twice? For instace: update t set text = 'abc' where id > = 1 . ISTM that the IV for OLD.text should be thrown away. > > Where would the key come from? Where would it be

Re: [HACKERS] 8.4.0 vs. locales vs. pl/perl?

2009-07-08 Thread Tom Lane
Andrew Gierth writes: > Looking at the code, the obvious thing that glares out is that the > locale setup in CheckMyDatabase is calling setlocale rather than > pg_perm_setlocale... am I missing something, or is this an obvious > bug? Sigh, it's an obvious bug. Whoever copied xlog.c's "pg_perm_se

Re: [HACKERS] 8.4.0 vs. locales vs. pl/perl?

2009-07-08 Thread Heikki Linnakangas
Andrew Gierth wrote: > environment: debian packaged 8.4.0 > > scenario: restoring a dump results in incorrect indexes for some > specific tables (indexscan order fails to match < comparison or > order resulting from explicit sorts). The dump contains plperl > and plperlu language creation and func

Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-08 Thread Alvaro Herrera
Brendan Jurd escribió: > Short answer: I could bring across the old commitfests but it would > take a couple hours at best per commitfest and result in little bits > of data loss here and there. I think we might be better off just > leaving the closed commitfests up on the wiki, and putting a not

[HACKERS] Does EXEC_BACKEND mode still need to propagate setlocale settings?

2009-07-08 Thread Tom Lane
The recent bug report about CheckMyDatabase() failing to use pg_perm_setlocale() led me to look for other uses of setlocale() that might be wrong. I found two fishy calls in restore_backend_variables(). I am thinking though that the correct fix is to delete them and get rid of the overhead of tran

Re: [HACKERS] Does EXEC_BACKEND mode still need to propagate setlocale settings?

2009-07-08 Thread Heikki Linnakangas
Tom Lane wrote: > The recent bug report about CheckMyDatabase() failing to use > pg_perm_setlocale() led me to look for other uses of setlocale() that > might be wrong. I found two fishy calls in restore_backend_variables(). I just did the exact same thing but you beat me.. > I am thinking thoug

Re: [HACKERS] Does EXEC_BACKEND mode still need to propagate setlocale settings?

2009-07-08 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> I am thinking though that the correct fix is to delete them and get >> rid of the overhead of transmitting the postmaster's setlocale settings >> to child processes this way. > Agreed. In EXEC_BACKEND case, main() sets them to the same system > defa

Re: [HACKERS] tsvector extraction patch

2009-07-08 Thread Alvaro Herrera
Mike Rylander escribió: > On Fri, Jul 3, 2009 at 3:49 AM, Hans-Juergen Schoenig -- > PostgreSQL wrote: > > test=# SELECT * FROM tsvcontent(to_tsvector('english', 'i am pretty sure > > this is a good patch')); > > lex   | rank > > +-- > > good   |    8 > > patch  |    9 > > pretti |    

Fwd: [HACKERS] tsvector extraction patch

2009-07-08 Thread Mike Rylander
Sorry, forgot to reply-all. -- Forwarded message -- From: Mike Rylander Date: Wed, Jul 8, 2009 at 4:17 PM Subject: Re: [HACKERS] tsvector extraction patch To: Alvaro Herrera On Wed, Jul 8, 2009 at 3:38 PM, Alvaro Herrera wrote: > Mike Rylander escribió: >> On Fri, Jul 3, 2009

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 7, 2009 at 6:33 PM, Tom Lane wrote: >> It's pretty much all-or-nothing now: the GUC does not give you any sort >> of useful control over *which* joins are reorderable. > Yes. So the way I see it, the options are: > 1. We can remove join_collapse_limit completel

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Kevin Grittner
Tom Lane wrote: > It occurs to me that one way to make GEQO less scary would be to > take out the nondeterminism by resetting its random number generator > for each query. You might get a good plan or an awful one, but at > least it'd be the same one each time. DBAs like predictability. +1

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Kenneth Marshall
On Wed, Jul 08, 2009 at 04:13:11PM -0500, Kevin Grittner wrote: > Tom Lane wrote: > > > It occurs to me that one way to make GEQO less scary would be to > > take out the nondeterminism by resetting its random number generator > > for each query. You might get a good plan or an awful one, but a

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Tom Lane
Noah Misch writes: > With joins between statistically indistinguishable columns, I see planning > times > change by a factor of ~4 for each join added or removed (postgres 8.3). > Varying > join_collapse_limit in the neighborhood of the actual number of joins has a > similar effect. See attach

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Tom Lane
"Kevin Grittner" writes: > For a moment it seemed logical to suggest a session GUC for the seed, > so if you got a bad plan you could keep rolling the dice until you got > one you liked; but my right-brain kept sending shivers down my spine > to suggest just how uncomfortable it was with that idea

[HACKERS] modules missing from Application Stack Wizard?

2009-07-08 Thread Kasia Tuszynska
Hello Postgres Hackers, We have begun testing Postgres 8.4 on windows, beginning with the installer. We have noticed that several additional modules which are usually installed through the Application Stack Wizard are missing from the list of available modules like PostGIS or the developer opti

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Kenneth Marshall
On Wed, Jul 08, 2009 at 05:46:02PM -0400, Tom Lane wrote: > "Kevin Grittner" writes: > > For a moment it seemed logical to suggest a session GUC for the seed, > > so if you got a bad plan you could keep rolling the dice until you got > > one you liked; but my right-brain kept sending shivers down

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Robert Haas
On Jul 8, 2009, at 3:57 PM, Tom Lane wrote: Robert Haas writes: On Tue, Jul 7, 2009 at 6:33 PM, Tom Lane wrote: It's pretty much all-or-nothing now: the GUC does not give you any sort of useful control over *which* joins are reorderable. Yes. So the way I see it, the options are: 1.

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Tom Lane
Robert Haas writes: > That was my first reaction too, but now I'm wondering whether we > shouldn't just do #1. #2 is a planner hint, too, just not a very good > one. If, as you suggest, it isn't actually useful, then why keep it > at all? (On the other hand, if someone thinks they need it,

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Itagaki Takahiro
Andrew Dunstan wrote: > I think you should have it use pthreads if available, or Windows threads > there, or fork() elsewhere. Just a question - which platform does not support any threading? I think threading is very common in modern applications. If there are such OSes, they seem to be just

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Joshua Tolley
On Wed, Jul 08, 2009 at 09:26:35PM -0400, Tom Lane wrote: > Robert Haas writes: > > That was my first reaction too, but now I'm wondering whether we > > shouldn't just do #1. #2 is a planner hint, too, just not a very good > > one. If, as you suggest, it isn't actually useful, then why keep

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Tom Lane
Joshua Tolley writes: > This sounds like planner hints to me. The argument against hinting, AIUI, is > that although the plan you've guaranteed via hints may be a good one today, > when the data change a bit your carefully crafted plan happens to become a bad > one, but you're no longer around to

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Tom Lane
Noah Misch writes: > Describing in those terms illuminates much. While the concepts do suggest 2^N > worst-case planning cost, my artificial test case showed a rigid 4^N pattern; > what could explain that? Well, the point of the 2^N concept is just that adding one more relation multiplies the pl

Re: [HACKERS] multi-threaded pgbench

2009-07-08 Thread Greg Smith
On Wed, 8 Jul 2009, Tom Lane wrote: pg_restore doesn't need anything more than a success/failure result from its child processes, but I think pgbench will want more. The biggest chunk of returned state to consider is how each client transaction generates a line of latency information that goe

Re: [HACKERS] Re: Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-08 Thread Fujii Masao
Hi, On Wed, Jul 8, 2009 at 10:59 PM, Kevin Grittner wrote: > Dimitri Fontaine wrote: > >>  4. sync: slave is no more lagging, it's applying the stream as it >>     gets it, either as part of the master transaction or not >>     depending on the GUC settings > > I think the interesting bit is when

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Robert Haas
On Jul 8, 2009, at 8:26 PM, Tom Lane wrote: Robert Haas writes: That was my first reaction too, but now I'm wondering whether we shouldn't just do #1. #2 is a planner hint, too, just not a very good one. If, as you suggest, it isn't actually useful, then why keep it at all? (On the other

[HACKERS] Round Robin Reviewers

2009-07-08 Thread Robert Haas
Folks, I am very pleased to announce that I have 13 volunteers to be Round-Robin Reviewers in addition to myself. Thanks to all who have stepped up to the plate. However, we have a lot more than 13 patches for this CommitFest, so more reviewers are still needed. In particular, we could use more