Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Gregory Stark wrote: The question I had was whether your solution for btree pointers marked dead and later dropped from the index works when the user hasn't configured a timeout and doesn't want standby queries killed. Yes, it's not any different from vacuum WAL reco

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Gregory Stark wrote: Simon Riggs writes: Increasing the waiting time increases the failover time and thus decreases the value of the standby as an HA system. Others value high availability higher than you and so we had agreed to provide an option to allow the max waiting time to be set. Sure

Re: [HACKERS] Is "Window" really safe as a global typedef name?

2008-12-19 Thread David Fetter
On Fri, Dec 19, 2008 at 10:36:30PM -0500, Tom Lane wrote: > I'm looking at the window-functions patch and wondering just what > kind of trouble we'll get into if we leave its new plan node type > named just "Window". I've already confirmed that this is a direct > conflict against a typedef in , an

Re: [HACKERS] [COMMITTERS] pgsql: SQL/MED catalog manipulation facilities This doesn't do any

2008-12-19 Thread Tom Lane
pet...@postgresql.org (Peter Eisentraut) writes: > Log Message: > --- > SQL/MED catalog manipulation facilities The buildfarm says this patch has got serious portability issues. grebe, dugong, and ermine are all crashing here: --- 109,115 (3 rows) ALTER FOREIGN DATA WRAPPER f

Re: [HACKERS] generic reloptions improvement

2008-12-19 Thread Alvaro Herrera
A small correction to this patch: this is needed because otherwise the autovac code to parse the option becomes all tangled; this avoids having to invent special values for "use the default value", and it also avoid having the default value stored elsewhere in the code than in the reloptions table.

[HACKERS] Is "Window" really safe as a global typedef name?

2008-12-19 Thread Tom Lane
I'm looking at the window-functions patch and wondering just what kind of trouble we'll get into if we leave its new plan node type named just "Window". I've already confirmed that this is a direct conflict against a typedef in , and I'd be not the least bit surprised if it's used in even-more-pop

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Bruce Momjian wrote: KaiGai Kohei wrote: but the auto-creation of pg_security rows does make this problematic. regproc works because it only supports pre-created pg_proc rows. Yes, the security label has a different characteristic at the point. Can we support two data types, one for read-only

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Bruce Momjian
KaiGai Kohei wrote: > > but the auto-creation of pg_security rows does make this problematic. > > regproc works because it only supports pre-created pg_proc rows. > > Yes, the security label has a different characteristic at the point. > > > Can we support two data types, one for read-only and a

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 19:29 -0500, Robert Treat wrote: > On Friday 19 December 2008 05:52:42 Simon Riggs wrote: > > BTW, I noticed the other day that Oracle 11g only allows you to have a > > read only slave *or* allows you to continue replaying. You need to > > manually switch back and forth betwe

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 20:54 +, Gregory Stark wrote: > "Kevin Grittner" writes: > > > PostgreSQL is much less prone to serialization failures, but it is > > certainly understandable if hot standby replication introduces new > > cases of it. > > In this case it will be possible to get this er

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Robert Treat
On Friday 19 December 2008 05:52:42 Simon Riggs wrote: > BTW, I noticed the other day that Oracle 11g only allows you to have a > read only slave *or* allows you to continue replaying. You need to > manually switch back and forth between those modes. They can't do > *both*, as Postgres will be able

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Alvaro Herrera
KaiGai Kohei wrote: >> Can we support two data types, one for read-only and another for >> possible creation? That is not going to work for a query like >> >> SELECT * FROM x WHERE col != >> 'valid_but_new_security_label'::regseclabel; >> >> because the cast would fail if the pg_security ro

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Here is one another idea. How do you think asymmetric input/output handlers? At first, I have to introduce the current implementation. It can receives user input values as TEXT or ACLITEM[], ant these are fetched at fetchWritableSystemAttribute() on execMain.c. Please note that they have no side-

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
but the auto-creation of pg_security rows does make this problematic. regproc works because it only supports pre-created pg_proc rows. Yes, the security label has a different characteristic at the point. Can we support two data types, one for read-only and another for possible creation? That

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
>>> Gregory Stark wrote: > I think the fundamental difference is that a deadlock or serialization > failure > can be predicted as a potential problem when writing the code. This is > something that can happen for any query any time, even plain old read-only > select queries. I've heard that

Re: [HACKERS] SQL standard definition shift for SERIALIZABLE

2008-12-19 Thread Kevin Grittner
>>> "Kevin Grittner" wrote: > Is an update in order? Never mind, it appears to have been covered already. Sorry for the noise. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hacker

[HACKERS] SQL standard definition shift for SERIALIZABLE

2008-12-19 Thread Kevin Grittner
I happened across this page: http://en.wikipedia.org/wiki/Isolation_(database_systems) I was all set to be righteously indignant over the slander of PostgreSQL, but figured I should check the SQL standard to be sure of my ground. The only version of the standard I have available right now is t

Re: [HACKERS] DTrace probes patch

2008-12-19 Thread Tom Lane
Robert Lor writes: > Tom Lane wrote: >> I agree. If the probe is meant to track only *some* WAL writes >> then it needs to be named something less generic than >> TRACE_POSTGRESQL_WAL_BUFFER_WRITE. >> > How about change it to TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DIRTY similar to > TRACE_POSTGRESQL

[HACKERS] generic reloptions improvement

2008-12-19 Thread Alvaro Herrera
Hi, Here's a patch for improving the general reloptions mechanism. What this patch does is add a table-based option parser. This allows adding new options very easily, and stops the business of having to pass the minimum and default fillfactor each time you want the reloptions processed. (This

Re: [HACKERS] DTrace probes patch

2008-12-19 Thread Robert Lor
Tom Lane wrote: "Fujii Masao" writes: I understood your intention. But, I think that its function name is somewhat confusing. I agree. If the probe is meant to track only *some* WAL writes then it needs to be named something less generic than TRACE_POSTGRESQL_WAL_BUFFER_WRITE. H

Re: [HACKERS] Windowing Function Patch Review -> Standard Conformance

2008-12-19 Thread Tom Lane
I've been studying the grammar for the windowing patch a bit. It seems to me that the option for got left out. I think that WindowDef needs to have two name fields, one for the name (if any) defined by the window definition, and one for the referenced window name if any. Also the "OVER name" s

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
>>> Gregory Stark wrote: > "Kevin Grittner" writes: > >> PostgreSQL is much less prone to serialization failures, but it is >> certainly understandable if hot standby replication introduces new >> cases of it. > > In this case it will be possible to get this error even if you're just > runnin

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
"Kevin Grittner" writes: > PostgreSQL is much less prone to serialization failures, but it is > certainly understandable if hot standby replication introduces new > cases of it. In this case it will be possible to get this error even if you're just running a single SELECT query -- and that's the

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
>>> Simon Riggs wrote: > The SQL Standard specifically names this error as thrown when "it > detects the inability to guarantee the serializability of two or more > concurrent SQL-transactions". Now that really should only apply when > running with SERIALIZABLE transactions, I disagree. Data

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
"Kevin Grittner" writes: Simon Riggs wrote: > >> max_standby_delay is set in recovery.conf, value 0 (forever) - > 2,000,000 >> secs, settable in milliseconds. So think of it like a deadlock > detector >> for recovery apply. > > Aha! A deadlock is a type of serialization failure. (In

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
Simon Riggs writes: > Increasing the waiting time increases the failover time and thus > decreases the value of the standby as an HA system. Others value high > availability higher than you and so we had agreed to provide an option > to allow the max waiting time to be set. Sure, it's a nice opt

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 13:47 -0600, Kevin Grittner wrote: > >>> Simon Riggs wrote: > > > max_standby_delay is set in recovery.conf, value 0 (forever) - > 2,000,000 > > secs, settable in milliseconds. So think of it like a deadlock > detector > > for recovery apply. > > Aha! A deadlock is a t

Re: [HACKERS] Windowing Function Patch Review -> Standard Conformance

2008-12-19 Thread Tom Lane
"Hitoshi Harada" writes: > Here's the patch, including latest function default args. Regards, The comments added to planner.c contain various claims that setrefs.c will fix up window function references, but I see no code related to that in setrefs.c. Please explain. reg

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
>>> Simon Riggs wrote: > max_standby_delay is set in recovery.conf, value 0 (forever) - 2,000,000 > secs, settable in milliseconds. So think of it like a deadlock detector > for recovery apply. Aha! A deadlock is a type of serialization failure. (In fact, on databases with lock-based concur

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 18:59 +, Gregory Stark wrote: > Simon Riggs writes: > > > The error message ought to be "snapshot too old", which could raise a > > chuckle, so I called it something else. > > > > The point you raise is a good one and I think we should publish a list > > of retryable er

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
Simon Riggs writes: > The error message ought to be "snapshot too old", which could raise a > chuckle, so I called it something else. > > The point you raise is a good one and I think we should publish a list > of retryable error messages. I contemplated once proposing a special log > level for a

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
>>> Simon Riggs wrote: > I understand the need, but we won't be using SQLSTATE = 40001. > > That corresponds to ERRCODE_T_R_SERIALIZATION_FAILURE, which that error > would not be. Isn't it a problem with serialization of database transactions? You hit it in a different way, but if it is a t

Re: [HACKERS] SQL/MED compatible connection manager

2008-12-19 Thread Martin Pihlak
Peter Eisentraut wrote: > I have committed this without the functions. I have some thoughts on > what to do about that, but right now I'm exhausted. ;-) > Great news :) Thanks a lot for your support and contributions! regards, Martin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 10:52 +, Simon Riggs wrote: > > You could > > conservatively use OldestXmin as latestRemovedXid, but that could stall > > the WAL redo a lot more than necessary. Or you could store > > latestRemovedXid in the page header, but that would need to be > > WAL-logged to

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 11:54 -0600, Kevin Grittner wrote: > >>> Simon Riggs wrote: > > > If I was going to add anything to the btree page header, it would be > > latestRemovedLSN, only set during recovery. That way we don't have > to > > explicitly kill queries, we can do the a wait on OldestXm

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
>>> Simon Riggs wrote: > If I was going to add anything to the btree page header, it would be > latestRemovedLSN, only set during recovery. That way we don't have to > explicitly kill queries, we can do the a wait on OldestXmin then let > them ERROR out when they find a page that has been modif

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Bruce Momjian
KaiGai Kohei wrote: > Gregory Stark wrote: > > KaiGai Kohei writes: > > > >> I didn't replace the previous implementation blindly, I have a few > >> reasons that the current one is better than previous one. > >> > >> For example, if an input handler has side-effects, what is happen in > >> the fo

Re: [HACKERS] SQL/MED compatible connection manager

2008-12-19 Thread Peter Eisentraut
I have committed this without the functions. I have some thoughts on what to do about that, but right now I'm exhausted. ;-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Gregory Stark wrote: KaiGai Kohei writes: I didn't replace the previous implementation blindly, I have a few reasons that the current one is better than previous one. For example, if an input handler has side-effects, what is happen in the following query? SELECT 'valid_but_new_security_la

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Gregory Stark
KaiGai Kohei writes: > I didn't replace the previous implementation blindly, I have a few > reasons that the current one is better than previous one. > > For example, if an input handler has side-effects, what is happen in > the following query? > > SELECT 'valid_but_new_security_label'::regse

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 09:22 -0500, Greg Stark wrote: > I'm confused shouldn't read-only transactions on the slave just be > hacked to not set any hint bits including lp_delete? They could be, though I see no value in doing so. But that is not Heikki's point. He is discussing what happens on

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Tom Lane wrote: > KaiGai Kohei writes: >> Tom Lane wrote: >>> This seems like a pretty bad idea that will eventually bite you in an >>> uncomfortable place. Lying about what datatype a field is is just not >>> safe. > >> Is it also correct for system attributes? >> I don't think the format on st

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Grzegorz Jaskiewicz
On 2008-12-19, at 13:07, Tom Lane wrote: Grzegorz Jaskiewicz writes: Filter: (NOT (hashed subplan)) ^^ If 8.3 does that, and 8.4 doesn't, it's most likely because you are using different work_mem settings. you're right, as always :) My point is, why is planner choos

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Greg Stark
I'm confused shouldn't read-only transactions on the slave just be hacked to not set any hint bits including lp_delete? -- Greg On 19 Dec 2008, at 03:49, Heikki Linnakangas > wrote: Whenever a B-tree index scan fetches a heap tuple that turns out to be dead, the B-tree item is marked as k

Re: [HACKERS] stat() vs cygwin

2008-12-19 Thread Andrew Dunstan
Bruce Momjian wrote: Andrew Dunstan wrote: Alvaro Herrera wrote: Andrew Dunstan wrote: I'm confused. There is a Cygwin member of buildfarm, working quite happily. Can you point me to the exact patch in question, please? I thought we resolved the matter of stat() ages ago

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Tom Lane
KaiGai Kohei writes: > Tom Lane wrote: >> This seems like a pretty bad idea that will eventually bite you in an >> uncomfortable place. Lying about what datatype a field is is just not >> safe. > Is it also correct for system attributes? > I don't think the format on storage has to be same as us

Re: [HACKERS] Simple postgresql.conf wizard

2008-12-19 Thread Tom Lane
"Mark Wong" writes: > On Mon, Dec 8, 2008 at 4:34 PM, Tom Lane wrote: >> "Mark Wong" writes: >>> On Tue, Dec 2, 2008 at 2:25 AM, Tom Lane wrote: Are any of the queries complicated enough to trigger GEQO planning? > Sorry for the delay in responding, here's the queries and the number > of

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 14:00 +0200, Heikki Linnakangas wrote: > Heikki Linnakangas wrote: > > Well, to be honest, I don't much like the whole notion of tracking the > > slots. I think we should just rely on the XLOG_RECOVERY_END records to > > purge stale PGPROC entries, belonging to backends tha

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Tom Lane
Grzegorz Jaskiewicz writes: >Filter: (NOT (hashed subplan)) ^^ If 8.3 does that, and 8.4 doesn't, it's most likely because you are using different work_mem settings. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Review: B-Tree emulation for GIN

2008-12-19 Thread Ibrar Ahmed
Thanks, On Fri, Dec 19, 2008 at 3:26 PM, Teodor Sigaev wrote: > Updated patch. > > Ibrar Ahmed wrote: > >> Hi Teodor Sigaev, >> >> I am getting server crash in contrib regression. May be I am doing >> something wrong here. Regression diff is attached. >> >> BTW these queries work fine outside t

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Well, to be honest, I don't much like the whole notion of tracking the slots. I think we should just rely on the XLOG_RECOVERY_END records to purge stale PGPROC entries, belonging to backends that died without writing an abort record. Sorry, I meant XLOG_XACT_RUNNING

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: On Fri, 2008-12-19 at 10:59 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: * corner case behaviour of backendids - bgwriter writes checkpoint WAL records. Has no backendid, but needs a slotid (possibly others) Why does bgwriter need a slotid? It doesn't run any transact

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 10:59 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > * corner case behaviour of backendids - bgwriter writes checkpoint WAL > > records. Has no backendid, but needs a slotid (possibly others) > > Why does bgwriter need a slotid? It doesn't run any transactions. >

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 12:24 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > We have infrastructure in place to make this work correctly, just need > > to add latestRemovedXid field to xl_btree_vacuum. So that part is easily > > solved. > > That's tricky because there's no xmin/xmax on i

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Grzegorz Jaskiewicz
and the plan on 8.4: gj=# explain select a.a from a where a not in (select a from b); QUERY PLAN - Seq Scan on a (cost=99035.00..257874197565.00 rows=300 width=4) Filter: (NOT (subplan))

Re: [HACKERS] Review: B-Tree emulation for GIN

2008-12-19 Thread Teodor Sigaev
Updated patch. Ibrar Ahmed wrote: Hi Teodor Sigaev, I am getting server crash in contrib regression. May be I am doing something wrong here. Regression diff is attached. BTW these queries work fine outside the regression. -- Ibrar Ahmed EnterpriseDB http://www.enterprisedb.com --

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: We have infrastructure in place to make this work correctly, just need to add latestRemovedXid field to xl_btree_vacuum. So that part is easily solved. That's tricky because there's no xmin/xmax on index tuples. You could conservatively use OldestXmin as latestRemovedXid, bu

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Grzegorz Jaskiewicz
true, but as soon as I drop indices on both tables: root=# explain analyze select a.a from a where a not in (select a from b); QUERY PLAN --

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 10:49 +0200, Heikki Linnakangas wrote: > Whenever a B-tree index scan fetches a heap tuple that turns out to be > dead, the B-tree item is marked as killed by calling _bt_killitems. When > the page gets full, all the killed items are removed by calling > _bt_vacuum_one_pa

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 11:04 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On a related but different point: We don't need an interlock between > > dirty buffers and WAL during recovery because the WAL has already been > > written. > > Assuming the WAL has also been fsync'd. True, so

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: On a related but different point: We don't need an interlock between dirty buffers and WAL during recovery because the WAL has already been written. Assuming the WAL has also been fsync'd. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsq

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: * corner case behaviour of backendids - bgwriter writes checkpoint WAL records. Has no backendid, but needs a slotid (possibly others) Why does bgwriter need a slotid? It doesn't run any transactions. * slotids are assigned once and never changed, so allowing them to be use

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 09:43 +0900, Fujii Masao wrote: > > Yes, please check the call points for ForceSyncCommit. > > > > Do I think every xlog flush should be synchronous, no, I don't. > That's why we have a user settable parameter for it. > > Umm.. I focus attention on XLogFlush() called except

[HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Whenever a B-tree index scan fetches a heap tuple that turns out to be dead, the B-tree item is marked as killed by calling _bt_killitems. When the page gets full, all the killed items are removed by calling _bt_vacuum_one_page. That's a problem for hot standby. If any of the killed b-tree ite

Re: [HACKERS] Summary: changes needed in function defaults behavior

2008-12-19 Thread Asko Oja
I would agree with making it stricter. It would force people to do less stupid things. Our main use case for default parameter will be getting rid of all the old versions of functions with shorter parameter lists by just creating new versions of old functions with additional default parameters. We