Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Marko Kreen
On 3/30/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Marko Kreen" <[EMAIL PROTECTED]> writes: > Now, why put it into core? I don't think you have made a sufficient case for that. I think it should stay as an outside project for awhile and see what sort of userbase it attracts. If it becomes suffic

Re: [HACKERS] UPDATE using sub selects

2007-03-30 Thread NikhilS
Hi, On 3/31/07, Tom Lane <[EMAIL PROTECTED]> wrote: NikhilS <[EMAIL PROTECTED]> writes: > I have invented a ROWEXPR_SUBLINK type that handles multiple output columns. > The trouble is that since eventually columns of the parents have to be part > of the query's targetList, I am sending the enti

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Tom Lane <[EMAIL PROTECTED]> wrote: I do not think you can assume that the plan won't be used later with some older snapshot. Consider recursive plpgsql functions for a counterexample: the inner occurrence might be the first to arrive at a given line of the function, hence the firs

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/31/07, Simon Riggs <[EMAIL PROTECTED]> wrote: On Fri, 2007-03-30 at 13:54 -0400, Tom Lane wrote: > > Hm. So anytime we reject a potentially useful index as being not valid > yet, we mark the plan as "only good for this top-level transaction"? > That seems possibly workable --- in particul

Re: [HACKERS] Modifying TOAST thresholds

2007-03-30 Thread Christopher Browne
[EMAIL PROTECTED] (Tom Lane) wrote: > I'm willing to do the code changes to separate TOAST_THRESHOLD from > the toast chunk size, but I do not have the time or facilities to do > any performance testing for different parameter choices. Anyone > want to work on that? What have you got in mind ther

Re: [HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-03-30 Thread Gregory Stark
Hm, my first thought was that you should just be using bind parameters instead of interpolating variables directly into the query. But the more I think about it the more I like your idea. It's true that using parameters takes away most of the use cases for the kind of interface you suggest. But t

Re: [HACKERS] [PATCHES] Full page writes improvement, code update

2007-03-30 Thread Koichi Suzuki
Simon; Tom; Koichi is writing. Your question is how to determine WAL record generated between pg_start_backup and pg_stop_backup and here's an answer. XLogInsert( ) already has a logic to determine if inserting WAL record is between pg_start_backup and pg_stop_backup. Currently it is used to

Re: [HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-03-30 Thread Tom Lane
<[EMAIL PROTECTED]> writes: > I'd like to see a new variant on PQexec(): > PGresult * PQexecf(PGconn *conn, const char *fmt, ...); Way too late for 8.3 --- if we were going to do something like this, we should think first and program later. In particular, blindly adopting the sprintf format s

[HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-03-30 Thread korryd
While cleaning up some pg_migrator code (http://pgfoundry.org/projects/pg-migrator/) it occurred to me that a typical libpq client application spends a lot of code constructing SQL commands. The code typically looks like this: a) allocate enough room to hold the command b) sp

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Florian G. Pflug
Simon Riggs wrote: On Fri, 2007-03-30 at 16:34 -0400, Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: 2. pg_stop_backup() should wait until all archive files are safely archived before returning Not sure I agree with that one. If it fails, you can't tell whether the action is done a

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 16:34 -0400, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > I'd like to make the following changes to recovery related code over the > > next few days/weeks. If anybody insists I do this by freeze or not at > > all, then I'll submit patches for 1,3,4,5,10 befo

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 16:35 -0400, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > Any page written during a backup has a backup block that would not be > > removable by Koichi's tool, so yes, you'd still be safe. > > How does it know not to do that? Not sure what you mean, but I'

Re: [HACKERS] Arrays of Complex Types

2007-03-30 Thread Tom Lane
David Fetter <[EMAIL PROTECTED]> writes: > After several rounds of patches, it appears that it might be easier to > create a new typtype entry, which I'll tentatively call 'a' because it > seems a little fragile and a lot inelegant and hard to maintain to > have typtype='c' and typrelid=InvalidOid

Re: [HACKERS] Oracle indemnifies PostgreSQL on its patents

2007-03-30 Thread Josh Berkus
Bruce, > Oracle, like IBM, Sony, RedHat, and Novell, is now a member of the Open > Invention Network, whose members all offer patent indemnification. Hey! We could go back to using ARC! ;-) -- Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)---

Re: [HACKERS] UPDATE using sub selects

2007-03-30 Thread Tom Lane
NikhilS <[EMAIL PROTECTED]> writes: > I have invented a ROWEXPR_SUBLINK type that handles multiple output columns. > The trouble is that since eventually columns of the parents have to be part > of the query's targetList, I am sending the entire subquery as one of the > entries in that list and the

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > Any page written during a backup has a backup block that would not be > removable by Koichi's tool, so yes, you'd still be safe. How does it know not to do that? regards, tom lane ---(end of broadcast)---

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > I'd like to make the following changes to recovery related code over the > next few days/weeks. If anybody insists I do this by freeze or not at > all, then I'll submit patches for 1,3,4,5,10 before Saturday night. I'd > rather take a bit more time and do

Re: [HACKERS] Arrays of Complex Types

2007-03-30 Thread David Fetter
On Fri, Mar 02, 2007 at 03:40:16PM -0800, David Fetter wrote: > Folks, > > I'd like to take the TODO item that reads, "Add support for arrays of > complex types," but before I start patching, I'd like to see whether > what I'm about to do makes any sense: After several rounds of patches, it appea

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Josh Berkus
Hannu, Marko, I, personally, think that it's worth talking about integrating these. However, the old versions were definitely NOT ready for integration, and the new versions went on the internet like a week ago. Heck, I haven't even downloaded them yet. Can we address these on the 8.4 timeli

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 13:54 -0400, Tom Lane wrote: > "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > > Actually, if we are using Serializable Snapshot then there is no chance > > to replan the query before the transaction completes and the next > > transaction to start in the session must see the in

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > Actually, if we are using Serializable Snapshot then there is no chance > to replan the query before the transaction completes and the next > transaction to start in the session must see the index and hence > we must replan. So it would be enough just

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/30/07, Florian G. Pflug <[EMAIL PROTECTED]> wrote: My idea was to store a list of xid's together with the cached plan that are assumed to be uncommitted accoring to the IndexSnapshot. The query is replanned if upon execution the IndexSnapshot assumes that one of these

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Tom Lane
"Marko Kreen" <[EMAIL PROTECTED]> writes: > Now, why put it into core? I don't think you have made a sufficient case for that. I think it should stay as an outside project for awhile and see what sort of userbase it attracts. If it becomes sufficiently popular I'd be willing to consider adding i

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Hmmm, remember that DatabaseCancelAutovacuumActivity is called on CREATE > DATABASE; but what it does is send SIGINT, not SIGTERM. Also, it's not > in 8.2. SIGINT does terminate the autovac process however. > I haven't read the whole problem report com

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Florian G. Pflug <[EMAIL PROTECTED]> wrote: My idea was to store a list of xid's together with the cached plan that are assumed to be uncommitted accoring to the IndexSnapshot. The query is replanned if upon execution the IndexSnapshot assumes that one of these xid's is committed.

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Alvaro Herrera
Tom Lane wrote: > I wrote: > > Heikki Linnakangas <[EMAIL PROTECTED]> writes: > >> statement_timeout interrupts seem to go through the PG_CATCH-block and > >> clean up the entry from the vacuum cycle array as they should. But a > >> SIGINT leading to a "terminating connection due to administrator

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Tom Lane
I wrote: > Heikki Linnakangas <[EMAIL PROTECTED]> writes: >> statement_timeout interrupts seem to go through the PG_CATCH-block and >> clean up the entry from the vacuum cycle array as they should. But a >> SIGINT leading to a "terminating connection due to administrator >> command" error does n

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Tom Lane wrote: I do not think you can assume that the plan won't be used later with some older snapshot. So maybe we'd need to use the SerializableSnapshot created at the start of each transaction for this check, and not the Act

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Tom Lane <[EMAIL PROTECTED]> wrote: That might work, but it doesn't seem to address the core objection: there's no mechanism to cause the query to be replanned once the snapshot is new enough, because no relcache inval will happen. So most likely existing backends will keep using

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 11:44 +0530, Pavan Deolasee wrote: > ISTM that the run-another-transaction-afterwards idea would have same > problem with plan invalidation. When the second transaction commits, > the relcache invalidation event is generated. The event may get > consumed > by other backends,

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I do not think you can assume that the plan won't be used later with >> some older snapshot. > So maybe we'd need to use the SerializableSnapshot created at the start > of each transaction for this check, and not the ActiveSnapsho

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
On 3/30/07, Florian G. Pflug <[EMAIL PROTECTED]> wrote: What about doing PREPARE myplan select ... ; outside of a transaction? Will this be execute inside a transaction? I checked that. PREPARE runs with ActiveSnapshot set. Thanks, Pavan -- EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: What I am suggesting is to use ActiveSnapshot (actually Florian's idea) to decide whether the transaction that created index was still running when we started. Isn't it the case that some snapshot will be "active" when we plan ? I do

[HACKERS] Oracle indemnifies PostgreSQL on its patents

2007-03-30 Thread Bruce Momjian
Here is a surprising article about how Oracle has made open source projects, like PostgreSQL, safe from claims of infringing Oracle patents: http://www.cbronline.com/article_news.asp?guid=A0F5F220-5940-470D-8564-CEA7E2D2B954. Oracle, like IBM, Sony, RedHat, and Novell, is now a member of

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > What I am suggesting is to use ActiveSnapshot (actually > Florian's idea) to decide whether the transaction that created > index was still running when we started. Isn't it the case that > some snapshot will be "active" when we plan ? I do not think y

Re: [HACKERS] [pgsql-www] Help: reading the source

2007-03-30 Thread Andrew Hammond
1) Wrong list, you want -hackers. 2) Did you have a specific question based on what you have read or are you asking for suggested reading? On 3/29/07, 李宁 <[EMAIL PROTECTED]> wrote: Dear everyone, I'm a college student,now I am doing the topic about the postgresql storage management as my the

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: How about storing the snapshot which we used during planning in CachedPlanSource, if at least one index was seen unusable because its CREATE INDEX transaction was seen as in-progress ? I'm getting tired of repeating this, but: the pl

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Pavan Deolasee
Tom Lane wrote: > > I'm getting tired of repeating this, but: the planner doesn't use a > snapshot. System catalogs run on SnapshotNow. > I am really sorry if I sound foolish here. I am NOT suggesting that we use "snapshot" to read system catalogs. I understand that system catalogs run on Snapsh

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > How about storing the snapshot which we used during planning in > CachedPlanSource, if at least one index was seen unusable because > its CREATE INDEX transaction was seen as in-progress ? I'm getting tired of repeating this, but: the planner doesn't

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > statement_timeout interrupts seem to go through the PG_CATCH-block and > clean up the entry from the vacuum cycle array as they should. But a > SIGINT leading to a "terminating connection due to administrator > command" error does not. Hm, that's

[HACKERS] Proposal: Adding CP51932 encoding

2007-03-30 Thread Tatsuo Ishii
Hi, Here is a proposal to add a client side encoding CP51932. CP51932 is an encoding for Japanese, defined by Microsoft. It is similar to existing EUC_JP encoding but some characters are mapped differently. Unfortunately for most Windows apps "EUC-JP" means CP51932. As a result, for example, if a

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Hannu Krosing
Ühel kenal päeval, R, 2007-03-30 kell 13:36, kirjutas Marko Kreen: > PL/Proxy is a small PL whose goal is to allow creating > "proxy functions" that call actual functions in remote database. > Basic design is: > > Function body describes how to deduce final database. Its either > > CONNECT 'con

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2007-03-30 at 11:27 +0100, Richard Huxton wrote: Is that always true? Could the backup not pick up a partially-written page? Assuming it's being written to as the backup is in progress. (We are talking about when disk blocks are smaller than PG blocks here, so can't

Re: [HACKERS] UPDATE using sub selects

2007-03-30 Thread NikhilS
Hi, No. Offhand I think you'd either need to relax EXPR_SUBLINK to allow > multiple output columns, or invent a ROW_SUBLINK SubLinkType that is > just like EXPR_SUBLINK except for allowing multiple output columns. > The latter would probably be less likely to break other things... > Yeah, was l

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 11:27 +0100, Richard Huxton wrote: > Is that always true? Could the backup not pick up a partially-written > page? Assuming it's being written to as the backup is in progress. (We > are talking about when disk blocks are smaller than PG blocks here, so > can't guarantee an

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Marko Kreen
On 3/30/07, Cédric Villemain <[EMAIL PROTECTED]> wrote: Le vendredi 30 mars 2007 12:36, Marko Kreen a écrit: > Patch: > > http://plproxy.projects.postgresql.org/plproxy_core.diff.gz Note a perhaps oversight in your makefile : + #REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-language=plpgsql --lo

Re: [HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Cédric Villemain
Le vendredi 30 mars 2007 12:36, Marko Kreen a écrit : > Patch: > > http://plproxy.projects.postgresql.org/plproxy_core.diff.gz Note a perhaps oversight in your makefile : + #REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-language=plpgsql --load-language=plproxy + REGRESS_OPTS = --dbname=regressio

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2007-03-30 at 10:22 +0200, Zeugswetter Andreas ADI SD wrote: Without a switch, because both full page writes and corresponding logical log is included in WAL, this will increase WAL size slightly (maybe about five percent or so). If everybody is happy with this, w

[HACKERS] Proposal: include PL/Proxy into core

2007-03-30 Thread Marko Kreen
PL/Proxy is a small PL whose goal is to allow creating "proxy functions" that call actual functions in remote database. Basic design is: Function body describes how to deduce final database. Its either CONNECT 'connstr'; -- connect to exactly this db or when partitioning is used: -- partito

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Zeugswetter Andreas ADI SD
> Archive recovery needs the > normal xlog record, which in some cases has been optimised > away because the backup block is present, since the full > block already contains the changes. Aah, I didn't know that optimization exists. I agree that removing that optimization is good/ok. Andreas

[HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Simon Riggs
I'd like to make the following changes to recovery related code over the next few days/weeks. If anybody insists I do this by freeze or not at all, then I'll submit patches for 1,3,4,5,10 before Saturday night. I'd rather take a bit more time and do this in one drop and there are some code dependen

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Simon Riggs
On Fri, 2007-03-30 at 10:22 +0200, Zeugswetter Andreas ADI SD wrote: > > Without a switch, because both full page writes and > > corresponding logical log is included in WAL, this will > > increase WAL size slightly > > (maybe about five percent or so). If everybody is happy > > with this, we

Re: [HACKERS] Autovacuum vs statement_timeout

2007-03-30 Thread Heikki Linnakangas
Tom Lane wrote: I seem to remember that we'd agreed that autovacuum should ignore any globally set statement_timeout, on the grounds that a poorly chosen setting could indefinitely prevent large tables from being vacuumed. But I do not see anything in autovacuum.c that resets the variable. Am I j

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-30 Thread Zeugswetter Andreas ADI SD
> Without a switch, because both full page writes and > corresponding logical log is included in WAL, this will > increase WAL size slightly > (maybe about five percent or so). If everybody is happy > with this, we > don't need a switch. Sorry, I still don't understand that. What is the "c