Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Pavel Stehule
2009/12/17 Andrew Dunstan : > > Recently there was discussion about allowing a COPY statement to be a SELECT > target, returning a text array, although the syntax wasn't really nailed > down that I recall. I was thinking that  we might have > >   COPY RETURNING ARRAY FROM ... > > instead of > >   C

Re: [HACKERS] Largeobject Access Controls (r2460)

2009-12-17 Thread Takahiro Itagaki
Robert Haas wrote: > In both cases, I'm lost. Help? They might be contrasted with the comments for myLargeObjectExists. Since we use MVCC visibility in loread(), metadata for large object also should be visible in MVCC rule. If I understand them, they say: * pg_largeobject_aclmask_snapshot

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread KaiGai Kohei
(2009/12/18 9:19), Tom Lane wrote: > KaiGai Kohei writes: >> [ patch to remove EnableDisableRule's permissions check ] > > I don't particularly like this patch, mainly because I disagree with > randomly removing permissions checks without any sort of plan about > where they ought to go. There ar

Re: [HACKERS] Largeobject Access Controls (r2460)

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 7:27 PM, Takahiro Itagaki wrote: >> > Another comment is I'd like to keep > > linkend="catalog-pg-largeobject-metadata"> >> > for the first pg_largeobject in each topic. >> Those two things aren't the same.  Perhaps you meant > linkend="catalog-pg-largeobject">? > Oops, yes

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 10:14 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> Disentangling that seems like a job and a half. > > Indeed it will be, but I think it would be a good thing to actually have > a defined point at which permissions checking is to be done. Comp

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Disentangling that seems like a job and a half. Indeed it will be, but I think it would be a good thing to actually have a defined point at which permissions checking is to be done. Trying to read the code and figure out what permissions you need to

Re: [HACKERS] LATERAL

2009-12-17 Thread Robert Haas
On Sun, Oct 18, 2009 at 2:57 PM, Tom Lane wrote: > Robert Haas writes: >> You could probably convince me that a merge join is not going to be >> too useful (how often can you want a merge join on the inner side of a >> nested loop? > > Why not?  As Andrew pointed out, what we're really trying to

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > KaiGai Kohei writes: > > [ patch to remove EnableDisableRule's permissions check ] > > I don't particularly like this patch, mainly because I disagree with > randomly removing permissions checks without any sort of plan about > where they ought to go. T

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-17 Thread Fujii Masao
On Thu, Dec 17, 2009 at 10:25 PM, Heikki Linnakangas wrote: > Fujii Masao wrote: >> I'm thinking of making the standby send the "walsender-switch-code" the same >> way >> as application_name; walreceiver always specifies the option like >> "replication=on" >> in conninfo string and calls PQconnec

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 7:19 PM, Tom Lane wrote: > If we're going to start moving these checks around we need a very > well-defined notion of where permissions checks should be made, so that > everyone knows what to expect.  I have not seen any plan for that. > Removing one check at a time because

Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost

2009-12-17 Thread Robert Haas
On Thu, Nov 26, 2009 at 4:25 PM, Robert Haas wrote: > On Sat, Nov 14, 2009 at 4:58 PM, Tom Lane wrote: >> I don't think there's even a >> solid consensus right now on which GUCs people would want to set at the >> tablespace level. > > This seems like an important point that we need to nail down.

Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost

2009-12-17 Thread Robert Haas
On Thu, Dec 3, 2009 at 11:00 AM, Robert Haas wrote: > On Sat, Nov 28, 2009 at 9:54 PM, David Rowley wrote: >> Robert Haas Wrote: >>> Hmm.  I'm not able to reliably detect a performance difference between >>> unpatched CVS HEAD (er... git master branch) and same with spcoptions- >>> v2.patch appli

Re: [HACKERS] Largeobject Access Controls (r2460)

2009-12-17 Thread Takahiro Itagaki
Robert Haas wrote: > > Another comment is I'd like to keep > linkend="catalog-pg-largeobject-metadata"> > > for the first pg_largeobject in each topic. > > Those two things aren't the same. Perhaps you meant linkend="catalog-pg-largeobject">? Oops, yes. Thank you for the correction. We als

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Joshua D. Drake
On Thu, 2009-12-17 at 12:16 -0600, Kevin Grittner wrote: > Tom Lane wrote: > > > After thinking a bit, I'd be inclined to add a new paragraph. > > In particular, now that FOR UPDATE actually works in subqueries, > > it'd be worth pointing out that you can add that to guard against > > this type

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Joshua D. Drake
On Thu, 2009-12-17 at 13:13 -0500, Tom Lane wrote: > "Joshua D. Drake" writes: > > What is needed here is a layman's context of what isolation modes are > > good for what type of operation. Neither your explanation or Tom's is > > particularly useful except to say, "Crap, I might be screwed but I

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread Tom Lane
KaiGai Kohei writes: > [ patch to remove EnableDisableRule's permissions check ] I don't particularly like this patch, mainly because I disagree with randomly removing permissions checks without any sort of plan about where they ought to go. There are two principal entry points in rewriteDefine.

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread KaiGai Kohei
(2009/12/18 6:38), Stephen Frost wrote: > KaiGai, > > * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: >> The patch was not attached... > > This patch either does too much, or not enough. > > I would either leave the Assert() in-place as a double-check (I presume > that's why it was there in the fir

Re: [HACKERS] [GENERAL] Installing PL/pgSQL by default

2009-12-17 Thread Bruce Momjian
Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian writes: > > > I installed PL/pgSQL by default via initdb with the attached patch. The > > > only problem is that pg_dump still dumps out the language creation: > > > CREATE PROCEDURAL LANGUAGE plpgsql; > > > ALTER PROCEDURAL LANGUAGE pl

Re: [HACKERS] PATCH: Spurious "22" in hstore.sgml

2009-12-17 Thread Greg Williamson
Bruce Momjian wrote: > > David Fetter wrote: > > On Thu, Dec 17, 2009 at 11:01:19AM +0100, Albe Laurenz wrote: > > > Magnus Hagander wrote: > > > > On Wed, Dec 16, 2009 at 20:34, David E. Wheeler wrote: > > > > > *** a/doc/src/sgml/hstore.sgml > > > > > --- b/doc/src/sgml/hstore.sgml > > > > >

Re: [HACKERS] An example of bugs for Hot Standby

2009-12-17 Thread Simon Riggs
On Wed, 2009-12-16 at 14:05 +, Simon Riggs wrote: > On Wed, 2009-12-16 at 10:33 +, Simon Riggs wrote: > > On Tue, 2009-12-15 at 20:25 +0900, Hiroyuki Yamada wrote: > > > Hot Standby node can freeze when startup process calls > > > LockBufferForCleanup(). > > > This bug can be reproduced by

Re: [HACKERS] [PATCH] remove redundant ownership checks

2009-12-17 Thread Stephen Frost
KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: > The patch was not attached... This patch either does too much, or not enough. I would either leave the Assert() in-place as a double-check (I presume that's why it was there in the first place, and if that Assert() fails then our assumption

Re: [HACKERS] PATCH: Spurious "22" in hstore.sgml

2009-12-17 Thread Bruce Momjian
David Fetter wrote: > On Thu, Dec 17, 2009 at 11:01:19AM +0100, Albe Laurenz wrote: > > Magnus Hagander wrote: > > > On Wed, Dec 16, 2009 at 20:34, David E. Wheeler wrote: > > > > *** a/doc/src/sgml/hstore.sgml > > > > --- b/doc/src/sgml/hstore.sgml > > > > > > Heh, interesting. That clearly shoul

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > [a transaction] might have also changed some other row so that it > now *does* satisfy WHERE, but we won't ever find that other row > because in the query snapshot it doesn't pass the WHERE. OK; got it. No way to fix that, really, without getting a fresh snapshot and re-star

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Dimitri Fontaine
Hi, Le 17 déc. 2009 à 19:39, Josh Berkus a écrit : > Mind you, returning (arbitrary expression) would be even better, but if > we can get returning TEXT[] for 8.5, I think it's worth doing on its own. Well, you already have it as soon as you have text[]: INSERT INTO destination SELECT row[0],

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 1:50 PM, Tom Lane wrote: > Robert Haas writes: >> You might want to specify column names as well as well as types, in >> this second case. > > Well, we could do it like VALUES: arbitrarily name the columns column1 > ... columnN and tell people to use an alias if they want

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Robert Haas wrote: > Don't get me wrong, I don't love the current behavior. (I don't > have a competing proposal either.) But I think we want to > describe it with precision, because there are also many cases > where _it works fine_. Telling people when it works and when it > doesn't work is a

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Kevin Grittner" writes: > Some quick testing seems to show that for the rows on which we were > blocking, all columns reflect all updates from the concurrent > transaction on which we were waiting, including columns used in the > WHERE clause. I'm not sure exactly what other tests might be > nec

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Tom Lane
Robert Haas writes: > You might want to specify column names as well as well as types, in > this second case. Well, we could do it like VALUES: arbitrarily name the columns column1 ... columnN and tell people to use an alias if they want other names. If it's convenient to fit column names into th

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
"Kevin Grittner" wrote: > Tom Lane wrote: > >> Are we sure that's a precise and complete description? I don't >> have a problem with putting a description just like that in the >> docs, but I'm not yet convinced it's right. > > Well, I thought it was when I typed it. You mentioned referenci

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 1:37 PM, Tom Lane wrote: > Andrew Dunstan writes: >> Tom Lane wrote: >>> The problem with COPY FROM is that it hard-wires a decision that there >>> is one and only one possible result format, which I think we pretty >>> much proved already is the wrong thing.  I'm not thri

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 1:12 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> I don't think that's any clearer, though it is more disparaging. >> :-) > > It's certainly not my goal to knock PostgreSQL.  The precise > conditions in which an UPDATE or DELETE can view an inconsistent > database s

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Josh Berkus
> In answer to Heiki's argument, what I wanted was exactly to return an > array of text for each row. Whatever we have needs to be able to handle > to possibility of ragged input (see previous discussion) so we can't tie > it down too tightly. I would have *lots* of use for this feature. Mind yo

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Tom Lane
Andrew Dunstan writes: > Tom Lane wrote: >> The problem with COPY FROM is that it hard-wires a decision that there >> is one and only one possible result format, which I think we pretty >> much proved already is the wrong thing. I'm not thrilled with "RETURNING >> ARRAY" either, but we need to le

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Andrew Dunstan
Tom Lane wrote: Heikki Linnakangas writes: Andrew Dunstan wrote: COPY RETURNING ARRAY FROM ... It's not really returning an array, is it? It's returning a bag of rows like a (sub)query. How about just COPY FROM? The problem with COPY FROM is that it har

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > Are we sure that's a precise and complete description? I don't > have a problem with putting a description just like that in the > docs, but I'm not yet convinced it's right. Well, I thought it was when I typed it. You mentioned referencing other columns in the updated rows

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Kevin Grittner" writes: > ... The precise > conditions in which an UPDATE or DELETE can view an inconsistent > database state (and therefore potentially persist something based on > that inconsistent state) are that it has a FROM clause and/or > subqueries which reference data changed by a concu

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > After thinking a bit, I'd be inclined to add a new paragraph. > In particular, now that FOR UPDATE actually works in subqueries, > it'd be worth pointing out that you can add that to guard against > this type of issue. Perhaps, after the "DELETE FROM website" > example, we cou

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Joshua D. Drake" writes: > What is needed here is a layman's context of what isolation modes are > good for what type of operation. Neither your explanation or Tom's is > particularly useful except to say, "Crap, I might be screwed but I don't > know if I am... how do I find out?" If we had a si

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Robert Haas wrote: > I don't think that's any clearer, though it is more disparaging. > :-) It's certainly not my goal to knock PostgreSQL. The precise conditions in which an UPDATE or DELETE can view an inconsistent database state (and therefore potentially persist something based on that i

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 1:05 PM, Tom Lane wrote: > "Kevin Grittner" writes: >> Tom Lane wrote: >>> I'm not very sure what a clearer explanation would look like > >> As a stab at it, how about?: > >> This behavior makes Read Committed mode unsuitable for many UPDATE >> or DELETE commands with joi

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> I'm not very sure what a clearer explanation would look like > As a stab at it, how about?: > This behavior makes Read Committed mode unsuitable for many UPDATE > or DELETE commands with joins or subqueries After thinking a bit, I'd be inclined t

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Joshua D. Drake
On Thu, 2009-12-17 at 12:58 -0500, Robert Haas wrote: > On Thu, Dec 17, 2009 at 12:51 PM, Kevin Grittner > wrote: > > Tom Lane wrote: > > > >> I'm not very sure what a clearer explanation would look like > > > > As a stab at it, how about?: > > > > This behavior makes Read Committed mode unsuitab

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 12:51 PM, Kevin Grittner wrote: > Tom Lane wrote: > >> I'm not very sure what a clearer explanation would look like > > As a stab at it, how about?: > > This behavior makes Read Committed mode unsuitable for many UPDATE > or DELETE commands with joins or subqueries I don'

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
Greg Stark writes: > I wonder if RETURNING hasn't created a whole new set of cases where > our READ COMMITTED behaviour is bogus. I don't see how. It just gives you access to the same values that were actually used by the UPDATE. regards, tom lane -- Sent via pgsql-hac

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > I'm not very sure what a clearer explanation would look like As a stab at it, how about?: This behavior makes Read Committed mode unsuitable for many UPDATE or DELETE commands with joins or subqueries -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Greg Stark
On Thu, Dec 17, 2009 at 5:39 PM, Tom Lane wrote: > Well, it would all depend on what you're trying to do.  Typical > single-row UPDATE commands aren't really affected by this problem, > and in fact the behavior is pretty much exactly what they want as > long as the WHERE conditions don't involve c

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Tom Lane
Heikki Linnakangas writes: > Andrew Dunstan wrote: >> COPY RETURNING ARRAY FROM ... > It's not really returning an array, is it? It's returning a bag of rows > like a (sub)query. > How about just COPY FROM? The problem with COPY FROM is that it hard-wires a decision that there is one and only o

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 12:38 PM, David Fetter wrote: > On Thu, Dec 17, 2009 at 12:28:50PM -0500, Robert Haas wrote: >> On Thu, Dec 17, 2009 at 12:23 PM, Heikki Linnakangas >> wrote: >> > How about just COPY FROM? As in >> > >> > SELECT t[5] as a, 3*(t[3]::numeric) as b FROM (COPY FROM STDIN >> >

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> http://developer.postgresql.org/pgdocs/postgres/transaction-iso.html#XACT-READ-COMMITTED > I don't know how many times I've read that page (many), yet I never > properly comprehended the impact of that part. I think the last bit > I quoted above is

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread David Fetter
On Thu, Dec 17, 2009 at 12:28:50PM -0500, Robert Haas wrote: > On Thu, Dec 17, 2009 at 12:23 PM, Heikki Linnakangas > wrote: > > How about just COPY FROM? As in > > > > SELECT t[5] as a, 3*(t[3]::numeric) as b FROM (COPY FROM STDIN > > CSV) as t > > I had the same thought. Though it would also b

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 12:23 PM, Heikki Linnakangas wrote: > How about just COPY FROM? As in > > SELECT t[5] as a, 3*(t[3]::numeric) as b FROM (COPY FROM STDIN CSV) as t I had the same thought. Though it would also be nice to allow something like: COPY (type1, type2, type3, type4) FROM STDIN C

Re: [HACKERS] COPY IN as SELECT target

2009-12-17 Thread Heikki Linnakangas
Andrew Dunstan wrote: > > Recently there was discussion about allowing a COPY statement to be a > SELECT target, returning a text array, although the syntax wasn't really > nailed down that I recall. I was thinking that we might have > >COPY RETURNING ARRAY FROM ... > > instead of > >C

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> the behavior under READ COMMITTED could be astonishing in certain >> circumstances as it breaks atomicity: > > Yup. That is stated fairly clearly already in the description of > READ COMMITTED mode, no? > http://developer.postgresql.org/pgdocs/postg

[HACKERS] COPY IN as SELECT target

2009-12-17 Thread Andrew Dunstan
Recently there was discussion about allowing a COPY statement to be a SELECT target, returning a text array, although the syntax wasn't really nailed down that I recall. I was thinking that we might have COPY RETURNING ARRAY FROM ... instead of COPY tablename opt_column_list FROM ...

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Kevin Grittner" writes: > Thanks for the clarification. That does not work for SERIALIZABLE > at all, because other tables or rows referenced in that first > statement would be using the original snapshot. Indeed, the > behavior under READ COMMITTED could be astonishing in certain > circumstanc

[HACKERS] Re: determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Greg Stark wrote: > So I for multi-statement transactions I don't see what this buys > you. Well, I became interested when Dr. Cahill said that adding this optimization yielded dramatic improvements in his high contention benchmarks. Clearly it won't help every load pattern. > You'll still h

Re: [HACKERS] Range types

2009-12-17 Thread Scott Bailey
Tom Lane wrote: Dimitri Fontaine writes: Tom Lane writes: Hm, how would you do it with LATERAL? The problem is not so much composition as the need for a variable number of rounds of composition. Let's have a try at it: select p2_member, array_accum(p1) from unnest(p2) as p2_member

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> we would instead get a fresh snapshot and retry -- which is what >> we do in a READ COMMITTED transaction. > I think you misunderstand how READ COMMITTED works; it does not > change the snapshot for the entire statement, it only follows the > update

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Greg Stark
On Thu, Dec 17, 2009 at 3:39 PM, Kevin Grittner wrote: > Basically, in a SERIALIZABLE transaction, if the first statement > which would require a snapshot would currently fail with "ERROR: > could not serialize access due to concurrent update" we would > instead get a fresh snapshot and retry -- w

Re: [HACKERS] Range types

2009-12-17 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Someone mentioned LATERAL? >> Tom Lane writes: >>> Hm, how would you do it with LATERAL? The problem is not so much >>> composition as the need for a variable number of rounds of >>> composition. >> Let's have a try at it: >> select p2_member, array_accu

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Tom Lane
"Kevin Grittner" writes: > Basically, in a SERIALIZABLE transaction, if the first statement > which would require a snapshot would currently fail with "ERROR: > could not serialize access due to concurrent update" we would > instead get a fresh snapshot and retry -- which is what we do in a > REA

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> "Markus Wanner" wrote: >> >>> Another line of thought: isn't this like READ COMMITTED for just >>> the first operation in a SERIALIZABLE transaction? >> >> I've mulled it over and I have two different logical proofs that >> this is safe; if anyone is

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 10:05 AM, Kevin Grittner wrote: > "Markus Wanner" wrote: > >> Another line of thought: isn't this like READ COMMITTED for just >> the first operation in a SERIALIZABLE transaction? > > I've mulled it over and I have two different logical proofs that > this is safe; if anyo

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
"Markus Wanner" wrote: > Another line of thought: isn't this like READ COMMITTED for just > the first operation in a SERIALIZABLE transaction? I've mulled it over and I have two different logical proofs that this is safe; if anyone is dubious I'd be happy to share. This seems likely to be of

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 13:38 +, Simon Riggs wrote: > > I see now that in the presence of prepared transactions, we would fail > > to clean up failed transations with XID > the oldest prepared > > transaction > > Good! I just spotted that also, just prior to posting my fix, so > rewriting it ag

Re: [HACKERS] Range types

2009-12-17 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> Hm, how would you do it with LATERAL? The problem is not so much >> composition as the need for a variable number of rounds of >> composition. > Let's have a try at it: > select p2_member, array_accum(p1) > from unnest(p2) as p2_member >

Re: [HACKERS] NOT IN Doesn't use Anti Joins?

2009-12-17 Thread Robert Haas
On Thu, Dec 17, 2009 at 9:02 AM, Rod Taylor wrote: > Is there a reason why the NOT IN plan could not use Anti-Joins when > the column being compared against is guaranteed to be NOT NULL? Too > much planner overhead to determine nullness of the column? I doubt it. I think it's just a question of

[HACKERS] NOT IN Doesn't use Anti Joins?

2009-12-17 Thread Rod Taylor
I'm sure there is a good reason why NOT IN will not use an Anti-Join plan equivalent to NOT EXISTS due to NULL handling, but in this particular case the value being compared is in the PRIMARY KEY of both structures being joined. The NOT IN plan was killed after 10 minutes. The NOT EXISTS plan retu

Re: [HACKERS] Largeobject Access Controls (r2460)

2009-12-17 Thread Robert Haas
2009/12/17 Takahiro Itagaki : > > Robert Haas wrote: > >> 2009/12/16 KaiGai Kohei : >> ? ?long desc: When turned on, privilege checks on large objects perform >> with >> ? ? ? ? ? ? ? backward compatibility as 8.4.x or earlier releases. > >> Mostly English quality, but there are so

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-17 Thread Kevin Grittner
Nicolas Barbier wrote: > Therefore, with next-key locking you better don't have too many table > scans if you want to have any concurrent transactions. Well, I would say that you don't want too many table scans on heavily updated tables if you don't want too many serialization failures. Keep in

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 15:18 +0200, Heikki Linnakangas wrote: > That removed piece of code was executed in the standby whenever we saw a > shutdown checkpoint. It calls ProcArrayApplyRecoveryInfo(), which calls > ExpireOldKnownAssignedTransactionIds() and StandbyReleaseOldLocks() to > clean up know

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-17 Thread Kevin Grittner
"Albe Laurenz" wrote: > I would say that in the case of a table scan, the whole table will > be SILOCKed. I guess that's pretty much unavoidable if you want > serializability. Agreed. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-17 Thread Florian Pflug
On 16.12.09 16:40 , Kevin Grittner wrote: Nicolas Barbier wrote: I am not sure whether the serialization failures that it may cause are dependent on the plan used. They are. But so are failures due to deadlocks even today, no? The processing order of UPDATES which involve joins isn't any

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
"Markus Wanner" wrote: > Another line of thought: isn't this like READ COMMITTED for just > the first operation in a SERIALIZABLE transaction? Hmmm... You have a point. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-17 Thread Heikki Linnakangas
Fujii Masao wrote: > I'm thinking of making the standby send the "walsender-switch-code" the same > way > as application_name; walreceiver always specifies the option like > "replication=on" > in conninfo string and calls PQconnectdb(), which sends the code as a part of > startup packet. And, the

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2009-12-17 at 13:24 +0200, Heikki Linnakangas wrote: > >> Hmm, looking at the code, I think Simon threw that baby with the >> bathwater when he removed support for starting standby from a shutdown >> checkpoint. > > Hmm, I think that code was just for starting points

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> Tom Lane wrote: >>> (Besides which the lock acquired by UPDATE isn't exclusive and >>> wouldn't block anyway...) >> It blocks other UPDATEs. > Not at the table level. The question was about whether we could change the timing of when we get the

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 13:24 +0200, Heikki Linnakangas wrote: > Hmm, looking at the code, I think Simon threw that baby with the > bathwater when he removed support for starting standby from a shutdown > checkpoint. Hmm, I think that code was just for starting points only. It would not have been e

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-17 Thread Fujii Masao
On Thu, Dec 17, 2009 at 9:02 PM, Heikki Linnakangas wrote: >> And, I think that such backend should switch to walsender mode when the >> startup >> packet arrives. Otherwise, we would have to authenticate such backend twice >> on different context, i.e., a normal backend and walsender. So the set

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-17 Thread Heikki Linnakangas
Fujii Masao wrote: > On Wed, Dec 16, 2009 at 6:53 PM, Heikki Linnakangas > wrote: >> 0. Begin by connecting to the master just like a normal backend does. We >> don't necessarily need the new ProtocolVersion code either, though it's >> probably still a good idea to reject connections to older serv

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 19:55 +0900, Hiroyuki Yamada wrote: > Well, I've read some more and have a question. > > The implementation assumes that transactions write COMMIT/ABORT WAL at the end > of them, while it does not seem to write ABORT WAL in immediate shutdown. So, > > 1. acquire ACCESS EXCL

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Heikki Linnakangas
Hiroyuki Yamada wrote: > The implementation assumes that transactions write COMMIT/ABORT WAL at the end > of them, while it does not seem to write ABORT WAL in immediate shutdown. So, > > 1. acquire ACCESS EXCLUSIVE lock in table A in xact 1 > 2. execute immediate shutdown of the active node > 3.

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Hiroyuki Yamada
>On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: > >> * There is a window beween gathering lock information in >> GetRunningTransactionLocks() >>and writing WAL in LogAccessExclusiveLocks(). >> * In current lock redo algorithm, locks are released when the transaction >> holding t

Re: [HACKERS] PATCH: Spurious "22" in hstore.sgml

2009-12-17 Thread David Fetter
On Thu, Dec 17, 2009 at 11:01:19AM +0100, Albe Laurenz wrote: > Magnus Hagander wrote: > > On Wed, Dec 16, 2009 at 20:34, David E. Wheeler wrote: > > > *** a/doc/src/sgml/hstore.sgml > > > --- b/doc/src/sgml/hstore.sgml > > > > Heh, interesting. That clearly shouldn't be there. Applied. > > Does

Re: [HACKERS] Hot Standby, release candidate?

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 12:01 +0200, Peter Eisentraut wrote: > On sön, 2009-12-13 at 19:20 +, Simon Riggs wrote: > > Barring resolving a few points and subject to even more testing, this > > is the version I expect to commit to CVS on Wednesday. > > So it's Thursday now. Please keep us updated

Re: [HACKERS] PATCH: Spurious "22" in hstore.sgml

2009-12-17 Thread Albe Laurenz
Magnus Hagander wrote: > On Wed, Dec 16, 2009 at 20:34, David E. Wheeler wrote: > > *** a/doc/src/sgml/hstore.sgml > > --- b/doc/src/sgml/hstore.sgml > > Heh, interesting. That clearly shouldn't be there. Applied. Does this count as catch-22? Yours, Laurenz Albe -- Sent via pgsql-hackers maili

Re: [HACKERS] Hot Standby, release candidate?

2009-12-17 Thread Peter Eisentraut
On sön, 2009-12-13 at 19:20 +, Simon Riggs wrote: > Barring resolving a few points and subject to even more testing, this > is the version I expect to commit to CVS on Wednesday. So it's Thursday now. Please keep us updated on the schedule, as we need to decide when to wrap alpha3 and whether

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-17 Thread Nicolas Barbier
[ Forgot the list, resending. ] 2009/12/16 Boszormenyi Zoltan : > Robert Haas írta: > >> On Wed, Dec 16, 2009 at 1:25 PM, Robert Haas wrote: >> >>> On Wed, Dec 16, 2009 at 1:14 PM, Alvaro Herrera >>> wrote: >>> So you'd have to disable HOT updates when true serializability was active?

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-17 Thread Albe Laurenz
Robert Haas wrote: > > A predicate can include columns from an index plus others. > > Am I missing something? > > Hmm, interesting point. In that case you couldn't use the index to > enforce predicate locking under MVCC without disabling HOT. But there > will be other cases where that wouldn't h

[HACKERS] How should the notice message from the primary be handled in the standby?

2009-12-17 Thread Fujii Masao
On Tue, Dec 15, 2009 at 12:56 AM, Tom Lane wrote: > Use PQsetNoticeReceiver.  The other one is just there for backwards > compatibility. Thanks! I'm thinking of changing nothing about handling of a notice message. Because there is no notice message (from the primary) which is worth being logged

Re: [HACKERS] Range types

2009-12-17 Thread Dimitri Fontaine
Tom Lane writes: > Dimitri Fontaine writes: >> Tom Lane writes: >>> foreach p2_member in unnest(p2) loop >>> p1 := array(select period_except(p1_member, p2_member) >>> from unnest(p1) p1_member); >>> end loop; >>> >>> But maybe it can be done in a sin

Re: [HACKERS] Streaming replication and non-blocking I/O

2009-12-17 Thread Fujii Masao
On Wed, Dec 16, 2009 at 6:53 PM, Heikki Linnakangas wrote: > Great! The logical next step is move the handling of TimelineID and > system identifier out of libpq as well. All right. > 0. Begin by connecting to the master just like a normal backend does. We > don't necessarily need the new Protoc

Re: [HACKERS] [BUG?] strange behavior in ALTER TABLE ... RENAME TO on inherited columns

2009-12-17 Thread KaiGai Kohei
It is a patch for the matter which I reported before. When a column is inherited from multiple relations, ALTER TABLE with RENAME TO option is problematic. This patch fixes the matter. In correctly, it prevent to rename columns inherited from multiple relations and merged. Also see the past discu

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Markus Wanner
Hi, Quoting "Tom Lane" : Not at the table level. If you could lock only at the tuple level maybe you'd have something AFAIUI this is about the tuple level lock, yes. but it seems like you can't find the target tuples without having acquired a snapshot. Maybe not *the* target tuple, but we