Re: [HACKERS] why can't plpgsql return a row-expression?

2012-12-05 Thread Pavel Stehule
2012/12/5 Tom Lane : > Asif Rehman writes: >> Here is the updated patch. I overlooked the loop, checking to free the >> conversions map. Here are the results now. > > I looked at this patch briefly. It seems to me to be completely > schizophrenic about the coercion rules. This bit: > > -

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-05 Thread Shigeru Hanada
On Mon, Nov 12, 2012 at 4:36 AM, Tomas Vondra wrote: > Hi, > > I've prepared a slightly updated patch, based on the previous review. > See it attached. All changes in v3 patch seem good, however I found some places which requires cosmetic changes. Please see attached v3.1 patch for those changes

Re: [HACKERS] strange isolation test buildfarm failure on guaibasaurus

2012-12-05 Thread Tom Lane
"Kevin Grittner" writes: > Stefan Kaltenbrunner wrote: >> seems like a rather odd failure in the isolation test (client) > Lines which might get the attention of interested parties: > TRAP: FailedAssertion("!(rel->rd_refcnt > 0)", File: "relcache.c", Line: 1603) Hah ... I'd looked at that report

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-05 Thread Amit Kapila
On Tuesday, December 04, 2012 8:37 AM Amit Kapila wrote: > On Monday, December 03, 2012 8:59 PM Tom Lane wrote: > > Robert Haas writes: > > > On Sat, Dec 1, 2012 at 11:45 AM, Tom Lane wrote: > > >>> But even if we can't make that work, it's not grounds for > reserving > > >>> PERSISTENT. Instead

[HACKERS] [BUG?] lag of minRecoveryPont in archive recovery

2012-12-05 Thread Kyotaro HORIGUCHI
Hello, I have a problem with PostgreSQL 9.2 with Pacemaker. HA standby sometime failes to start under normal operation. Testing with a bare replication pair showed that the standby failes startup recovery under the operation sequence shown below. 9.3dev too, but 9.1 does not have this problem. Th

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-12-05 Thread Jan Wieck
On 12/5/2012 2:00 PM, Robert Haas wrote: Many it'd be sensible to relate the retry time to the time spend vacuuming the table. Say, if the amount of time spent retrying exceeds 10% of the time spend vacuuming the table, with a minimum of 1s and a maximum of 1min, give up. That way, big tables w

[HACKERS] Fix for pg_upgrade status display

2012-12-05 Thread Bruce Momjian
Pg_upgrade displays file names during copy and database names during dump/restore. Andrew Dunstan identified three bugs: * long file names were being truncated to 60 _leading_ characters, which often do not change for long file names * file names were truncated to 60 characters in log files *

Re: [HACKERS] Forgotten argument description in header of index_create

2012-12-05 Thread Alvaro Herrera
Michael Paquier escribió: > Hi all, > > While reading some index-related code, I found that the description of the > argument is_internal of index_create in index.c has been forgotten in > commit f4c4335. > Correction patch attached. Thanks, pushed. -- Álvaro Herrerahttp://www.2

Re: [HACKERS] WIP: store additional info in GIN index

2012-12-05 Thread Tomas Vondra
On 5.12.2012 09:10, Alexander Korotkov wrote: > On Wed, Dec 5, 2012 at 1:56 AM, Tomas Vondra > wrote: > > Thanks for bug report. It is fixed in the attached patch. Hi, I gave it another try and this time it went fine - I didn't get any segfault when loading the data, which

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-05 Thread Jeff Davis
On Wed, 2012-12-05 at 19:43 -0500, Tom Lane wrote: > From memory, the tqual.c code assumes that any tuple with XMIN_COMMITTED > couldn't possibly be from its own transaction, and thus it doesn't make > the tests that would be appropriate for a tuple that is from the current > transaction. Maybe it

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 6:45 PM, Simon Riggs wrote: > Yes, but it is also the trigger writers problem. Maybe to some degree. I don't think that a server crash or something like a block-read error is ever tolerable though, no matter how silly the user is with their event trigger logic. If we go d

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-05 Thread Tom Lane
Robert Haas writes: > On Tue, Dec 4, 2012 at 3:38 PM, Jeff Davis wrote: >> After reading that thread, I still don't understand why it's unsafe to >> set HEAP_XMIN_COMMITTED in those conditions. Even if it is, I would >> think that a sufficiently narrow case -- such as CTAS outside of a >> transac

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: > In the patch we're talking about, the --extension-script is an > accumulative option that needs an argument, so you do >pg_dump --extension-script istore --extension-script foo > or if you're into short options >pg_dump -X istore -X foo -X bar My reaction to

Re: [HACKERS] Fwd: question on foreign key lock

2012-12-05 Thread Tom Lane
Robert Haas writes: > On Wed, Dec 5, 2012 at 7:08 AM, Filip Rembiałkowski > wrote: >> 3. I made a naive test of simply changing AccessExclusiveLock to >> ExclusiveLock, and seeing how many regression tests it breaks. It >> breaks none :-) > Sure. You could probably downgrade it quite a bit fur

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: > Robert Haas writes: >> Well, there's certainly a point, because IIUC Dimitri's patch dumps >> the file into the pg_dump output no matter whether the file originally >> came from an SQL command or the filesystem. IMHO, anyone who thinks >> that isn't going to break thin

Re: [HACKERS] Enabling Checksums

2012-12-05 Thread Simon Riggs
On 5 December 2012 23:40, Robert Haas wrote: > On Tue, Dec 4, 2012 at 6:17 PM, Jeff Davis wrote: >> Or, I could write up a test framework in ruby or python, using the >> appropriate pg driver, and some not-so-portable shell commands to start >> and stop the server. Then, I can publish that on thi

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Robert Haas writes: > In other words, the first paragraph is arguing for something like the > notion of an extension template - the ability to store the extension > files inside the server, in cases where you don't want them to appear > in the file system. But perhaps implemented using functions

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-05 Thread Robert Haas
On Tue, Dec 4, 2012 at 3:38 PM, Jeff Davis wrote: > After reading that thread, I still don't understand why it's unsafe to > set HEAP_XMIN_COMMITTED in those conditions. Even if it is, I would > think that a sufficiently narrow case -- such as CTAS outside of a > transaction block -- would be safe

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Simon Riggs
On 5 December 2012 23:34, Robert Haas wrote: > On Wed, Dec 5, 2012 at 5:56 PM, Andres Freund wrote: >> On 2012-12-05 22:41:21 +, Simon Riggs wrote: >>> On 5 December 2012 22:21, Dimitri Fontaine wrote: >>> > Simon Riggs writes: >>> >>> CREATE EVENT TRIGGER my_event_trigger >>> >>> ON ta

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Simon Riggs
On 5 December 2012 22:49, Dimitri Fontaine wrote: > Simon Riggs writes: >> On 5 December 2012 22:21, Dimitri Fontaine wrote: >>> Simon Riggs writes: > CREATE EVENT TRIGGER my_event_trigger > ON table_rewrite > EXECUTE PROCEDURE consider_whether_to_throw_an_error();

Re: [HACKERS] Enabling Checksums

2012-12-05 Thread Robert Haas
On Tue, Dec 4, 2012 at 6:17 PM, Jeff Davis wrote: > Or, I could write up a test framework in ruby or python, using the > appropriate pg driver, and some not-so-portable shell commands to start > and stop the server. Then, I can publish that on this list, and that > would at least make it easier to

[HACKERS] Functional dependency in GROUP BY through JOINs

2012-12-05 Thread David Rowley
Some idle thoughts has provoked me to sending this email. I'm wondering what I'm seeing here, is a simple missed optimisation or something that would be very difficult to implement, or even perhaps something I've completely misunderstood. To set the scene let's go with the old products and sale

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Robert Haas writes: > Well, with the design you have proposed, unless you have access to the > filesystem, it ain't gonna work. And if you have access to the > filesystem, then this whole discussion is moot. I did mention that this version of the patch is only ready to host the current design ta

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:56 PM, Andres Freund wrote: > On 2012-12-05 22:41:21 +, Simon Riggs wrote: >> On 5 December 2012 22:21, Dimitri Fontaine wrote: >> > Simon Riggs writes: >> >>> CREATE EVENT TRIGGER my_event_trigger >> >>> ON table_rewrite >> >>> EXECUTE PROCEDURE consider_whe

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:43 PM, Dimitri Fontaine wrote: >> I don't find that argument convincing in the slightest. Could I perhaps >> convince you to dig up a reference? I would be interested in the >> arguments for that design back then. > > I think here it is: > > http://archives.postgresql.or

Re: [HACKERS] strange isolation test buildfarm failure on guaibasaurus

2012-12-05 Thread Kevin Grittner
Stefan Kaltenbrunner wrote: > Subject: [HACKERS] strange isolation test buildfarm failure on guaibasaurus > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=guaibasaurus&dt=2012-12-05%2016%3A17%3A01 > seems like a rather odd failure in the isolation test (client) Lines which might get the atte

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:08 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> Yeah, DROP will work. But what about ALTER .. UPDATE? > > What about it? Well, with the design you have proposed, unless you have access to the filesystem, it ain't gonna work. And if you have access to the filesy

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Andres Freund
On 2012-12-05 22:41:21 +, Simon Riggs wrote: > On 5 December 2012 22:21, Dimitri Fontaine wrote: > > Simon Riggs writes: > >>> CREATE EVENT TRIGGER my_event_trigger > >>> ON table_rewrite > >>> EXECUTE PROCEDURE consider_whether_to_throw_an_error(); > >> > >> +1, I was just thinking t

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Dimitri Fontaine
Simon Riggs writes: > On 5 December 2012 22:21, Dimitri Fontaine wrote: >> Simon Riggs writes: CREATE EVENT TRIGGER my_event_trigger ON table_rewrite EXECUTE PROCEDURE consider_whether_to_throw_an_error(); >>> >>> +1, I was just thinking that myself. >> >> +1, and I think

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Andres Freund writes: >> The argument here is that the user would then have packaged its >> extension as files in the meantime. If not, that's operational error. A >> backup you didn't restore successfully isn't a backup anyway. > > Uh. Wait. What? If that argument is valid, we don't need anything

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Simon Riggs
On 5 December 2012 22:21, Dimitri Fontaine wrote: > Simon Riggs writes: >>> CREATE EVENT TRIGGER my_event_trigger >>> ON table_rewrite >>> EXECUTE PROCEDURE consider_whether_to_throw_an_error(); >> >> +1, I was just thinking that myself. > > +1, and I think that can happen for 9.3, as soo

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 23:28:45 +0100, Dimitri Fontaine wrote: > Alvaro Herrera writes: > > What happens on a normal pg_dump of the complete database? For > > extensions that were installed using strings instead of files, do I get > > a string back? Because if not, the restore is clearly going to fail >

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Simon Riggs
On 5 December 2012 22:22, Andres Freund wrote: > On 2012-12-05 19:03:44 -0300, Alvaro Herrera wrote: >> Simon Riggs wrote: >> > Prefer >> > BgWorkerStart_ConsistentState to be renamed to BgWorkerRun_InHotStandby >> > BgWorkerStart_RecoveryFinished to be renamed to BgWorkerRun_InNormalMode >> > >>

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Alvaro Herrera writes: > What happens on a normal pg_dump of the complete database? For > extensions that were installed using strings instead of files, do I get > a string back? Because if not, the restore is clearly going to fail > anyway. The argument here is that the user would then have pa

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Andres Freund
On 2012-12-05 18:42:42 -0300, Alvaro Herrera wrote: > > > > bgw_sighup and bgw_sigterm > > are > > pointers to functions that will be installed as signal handlers for the > > new > > - process. > > + process. XXX: Can they be NULL? > > > > Hm. The code doesn't check, so what

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Andres Freund
On 2012-12-05 19:03:44 -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > Prefer > > BgWorkerStart_ConsistentState to be renamed to BgWorkerRun_InHotStandby > > BgWorkerStart_RecoveryFinished to be renamed to BgWorkerRun_InNormalMode > > > > presumably a process will shutdown if (BgWorkerRun_InH

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Dimitri Fontaine
Simon Riggs writes: >> CREATE EVENT TRIGGER my_event_trigger >> ON table_rewrite >> EXECUTE PROCEDURE consider_whether_to_throw_an_error(); > > +1, I was just thinking that myself. +1, and I think that can happen for 9.3, as soon as we agree on the list of code points where we want that e

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Alvaro Herrera
Dimitri Fontaine escribió: > Robert Haas writes: > > Well, there's certainly a point, because IIUC Dimitri's patch dumps > > the file into the pg_dump output no matter whether the file originally > > came from an SQL command or the filesystem. IMHO, anyone who thinks > > that isn't going to brea

Re: [HACKERS] Fwd: question on foreign key lock

2012-12-05 Thread Andres Freund
On 2012-12-05 17:05:41 -0500, Robert Haas wrote: > On Wed, Dec 5, 2012 at 7:08 AM, Filip Rembiałkowski > wrote: > > Robert, thank you for the answer. > > > > 1. "need exclusive lock anyway to add triggers". > > Why adding a trigger needs exclusive lock? > > Someone would say blocking reads is not

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Robert Haas writes: > Yeah, DROP will work. But what about ALTER .. UPDATE? What about it? > Well, there's certainly a point, because IIUC Dimitri's patch dumps > the file into the pg_dump output no matter whether the file originally > came from an SQL command or the filesystem. IMHO, anyone w

Re: [HACKERS] Fwd: question on foreign key lock

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 7:08 AM, Filip Rembiałkowski wrote: > Robert, thank you for the answer. > > 1. "need exclusive lock anyway to add triggers". > Why adding a trigger needs exclusive lock? > Someone would say blocking reads is not needed (since possible trigger > events are: Insert/Update/Del

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Alvaro Herrera
Simon Riggs wrote: > On 5 December 2012 15:09, Alvaro Herrera wrote: > > Here's a first attempt at a new documentation chapter. This goes in > > part "Server Programming", just after the SPI chapter. > > > > I just noticed that worker_spi could use some more sample code, for > > example auth_coun

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 2:54 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> I have no objection whatsoever to the concept of storing the SQL and >> control files somewhere that doesn't need access to the server >> filesystem - in fact, I think I previously proposed allowing those to >> be sto

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 16:42:38 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-05 16:20:41 -0500, Tom Lane wrote: > >> GUC or no GUC, it'd still be letting an unprivileged network-exposed > >> application (PG) do something that's against any sane system-level > >> security policy. Lipstick

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Simon Riggs
On 5 December 2012 15:09, Alvaro Herrera wrote: > Here's a first attempt at a new documentation chapter. This goes in > part "Server Programming", just after the SPI chapter. > > I just noticed that worker_spi could use some more sample code, for > example auth_counter was getting its own LWLock

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Alvaro Herrera
Andres Freund wrote: > On 2012-12-05 12:09:58 -0300, Alvaro Herrera wrote: > > Here's a first attempt at a new documentation chapter. This goes in > > part "Server Programming", just after the SPI chapter. > > > > I just noticed that worker_spi could use some more sample code, for > > example auth

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Andres Freund writes: > On 2012-12-05 16:20:41 -0500, Tom Lane wrote: >> GUC or no GUC, it'd still be letting an unprivileged network-exposed >> application (PG) do something that's against any sane system-level >> security policy. Lipstick is not gonna help this pig. > What about the non-writab

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 16:20:41 -0500, Tom Lane wrote: > Dimitri Fontaine writes: > >> On 2012-12-05 13:18:16 -0500, Tom Lane wrote: > >>> I think you're wasting your time to imagine that that case will ever be > >>> "fixed". Allowing the server to scribble on executable files would set > >>> off all kind

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-12-05 Thread Andres Freund
On 2012-12-05 12:09:58 -0300, Alvaro Herrera wrote: > Here's a first attempt at a new documentation chapter. This goes in > part "Server Programming", just after the SPI chapter. > > I just noticed that worker_spi could use some more sample code, for > example auth_counter was getting its own LWLo

[HACKERS] strange isolation test buildfarm failure on guaibasaurus

2012-12-05 Thread Stefan Kaltenbrunner
Hi all! http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=guaibasaurus&dt=2012-12-05%2016%3A17%3A01 seems like a rather odd failure in the isolation test (client) Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.p

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: >> On 2012-12-05 13:18:16 -0500, Tom Lane wrote: >>> I think you're wasting your time to imagine that that case will ever be >>> "fixed". Allowing the server to scribble on executable files would set >>> off all kinds of security alarm bells, and rightly so. If Postgres

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Andres Freund writes: > On 2012-12-05 21:16:52 +0100, Dimitri Fontaine wrote: >> Now if we can't fix the executable files situation, what about making >> the C coded extensions not require an executable anymore? I'm thinking >> about studying what it would take exactly to write a PL/C where the >>

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Alvaro Herrera writes: > +1. Certainly a pg_dump patch's thread is not the place to propose it. Sure. Sorry about that, the goal of that previous message was to let people come to understand better my whole vision of what is an Extension, a contrib, and where we are what I wanted us to build. I

Re: [HACKERS] why can't plpgsql return a row-expression?

2012-12-05 Thread Tom Lane
Asif Rehman writes: > Here is the updated patch. I overlooked the loop, checking to free the > conversions map. Here are the results now. I looked at this patch briefly. It seems to me to be completely schizophrenic about the coercion rules. This bit: - if (atttypid != at

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Alvaro Herrera
Andres Freund escribió: > On 2012-12-05 21:16:52 +0100, Dimitri Fontaine wrote: > > Now if we can't fix the executable files situation, what about making > > the C coded extensions not require an executable anymore? I'm thinking > > about studying what it would take exactly to write a PL/C where t

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Simon Riggs
On 5 December 2012 19:15, Robert Haas wrote: > On Wed, Dec 5, 2012 at 1:41 PM, Josh Berkus wrote: >> That's why we need a mechanism which either logs, or aborts on specific >> actions. From the perspective of the DevOps staff, abort is possibly >> the better option, but there may be issues with

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
So, Lots of things are being said, most of them are really interesting and some of them are just re-hashing what we said about a year ago in the "Inline Extensions" thread, whose conclusion was that the key not to have two different beasts (inline, file based) was for pg_dump to process them all t

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 21:16:52 +0100, Dimitri Fontaine wrote: > Andres Freund writes: > > Adding some *NON WRITABLE* per-cluster library directory doesn't seem to > > be as controversion as other suggestions. > > Well, it means a per-initdb (user driven) location where to store binary > files, ask Tom wha

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Andres Freund writes: > Maybe I am missing something, but you already can separate them per > major version. You co-wrote the debian infrastructure to do so for some > debian packages, so I am not sure what you mean here. The debian infrastructure I've help building is all about compiling an exte

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Alvaro Herrera
Robert Haas escribió: > I have no objection whatsoever to the concept of storing the SQL and > control files somewhere that doesn't need access to the server > filesystem - in fact, I think I previously proposed allowing those to > be stored in a database table. You could do that with something l

Re: [HACKERS] json accessors

2012-12-05 Thread David E. Wheeler
On Dec 5, 2012, at 11:51 AM, Andrew Dunstan wrote: >> So I'm happy with this stuff, as long as it does not get in the way of >> supporting indexing at some point in the future. I can’t wait to start using >> it! > > I don't see why it should get in the way of anything like that. If anything,

Re: [HACKERS] json accessors

2012-12-05 Thread Merlin Moncure
On Wed, Dec 5, 2012 at 12:42 PM, David E. Wheeler wrote: > On Dec 5, 2012, at 9:57 AM, Merlin Moncure wrote: > >> Indexing large documents for fancy querying is a niche case but also >> quite complex. This isn't very well covered by xmlpath either btw -- >> I think for inspiration we should be l

Re: [HACKERS] json accessors

2012-12-05 Thread Andrew Dunstan
On 12/05/2012 01:48 PM, David E. Wheeler wrote: I'm sorry what I have offered isn't what you want, but plenty of other people have told me it will go a long way meeting their needs. *Sigh.* I guess I have not been clear. The stuff you propose is *awesome*. I love it. The syntax with the chain

Re: [HACKERS] json accessors

2012-12-05 Thread Andrew Dunstan
On 12/05/2012 01:49 PM, Josh Berkus wrote: *) xmlpath/jsonpath do searching (and decomposition) but are very clunky from sql perspective and probably absolutely nogo in terms if GIST/GIN. postgres spiritually wants to do things via operators and we should (if possible) at least consider that fi

Re: [HACKERS] json accessors

2012-12-05 Thread Andres Freund
On 2012-12-05 10:49:35 -0800, Josh Berkus wrote: > > >> *) xmlpath/jsonpath do searching (and decomposition) but are very > >> clunky from sql perspective and probably absolutely nogo in terms if > >> GIST/GIN. postgres spiritually wants to do things via operators and > >> we should (if possible)

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Josh Berkus
> You know, event triggers seem like an awfully good solution to this > problem. All we'd need is a new event called table_rewrite: > > CREATE EVENT TRIGGER my_event_trigger > ON table_rewrite > EXECUTE PROCEDURE consider_whether_to_throw_an_error(); Oh, wow, that would be perfect. Tha

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 2:19 PM, Andres Freund wrote: >> IIUC, under this proposal, the client would instead need to execute >> some SQL code that looks something this (I'm faking the syntax here, >> forgive me, but the patch doesn't seem to contemplate ALTER): >> >> ALTER EXTENSION myextension UPD

Re: [HACKERS] json accessors

2012-12-05 Thread Merlin Moncure
On Wed, Dec 5, 2012 at 12:49 PM, Josh Berkus wrote: > >>> *) xmlpath/jsonpath do searching (and decomposition) but are very >>> clunky from sql perspective and probably absolutely nogo in terms if >>> GIST/GIN. postgres spiritually wants to do things via operators and >>> we should (if possible)

Re: [HACKERS] Switching timeline over streaming replication

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch, which just add some sanity checks to notice incorrect PITR scenarios. Here'

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 14:10:34 -0500, Robert Haas wrote: > On Wed, Dec 5, 2012 at 2:01 PM, Andres Freund wrote: > > E.g. for years I had a set of (trigger) functions to counted the number > > of rows in a table in a lockless manner. That's used in 10+ applications > > of former clients of mine. All (plpg)

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 1:41 PM, Josh Berkus wrote: > That's why we need a mechanism which either logs, or aborts on specific > actions. From the perspective of the DevOps staff, abort is possibly > the better option, but there may be issues with it on our end. That was > the attraction of the or

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 2:01 PM, Andres Freund wrote: > E.g. for years I had a set of (trigger) functions to counted the number > of rows in a table in a lockless manner. That's used in 10+ applications > of former clients of mine. All (plpg)sql. > Imagine I want to ship an updated version that 1.

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 13:50:27 -0500, Robert Haas wrote: > On Wed, Dec 5, 2012 at 12:47 PM, Andres Freund wrote: > > For me it seems pretty natural to support dumping extension the way they > > got created. I.e. a plain CREATE EXTENSION ...; if the extension was > > preinstalled and some form that include

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 11:24 AM, Kevin Grittner wrote: > Robert Haas wrote: >> Since people *already* raise deadlock_timeout to obscenely high >> values (a minute? an hour???) and then complain that things blow >> up in their face, I think there's a decent argument to be made >> that piggybacking

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 12:47 PM, Andres Freund wrote: > For me it seems pretty natural to support dumping extension the way they > got created. I.e. a plain CREATE EXTENSION ...; if the extension was > preinstalled and some form that includes the extension source if you > installed it via the conn

Re: [HACKERS] json accessors

2012-12-05 Thread Josh Berkus
>> *) xmlpath/jsonpath do searching (and decomposition) but are very >> clunky from sql perspective and probably absolutely nogo in terms if >> GIST/GIN. postgres spiritually wants to do things via operators and >> we should (if possible) at least consider that first Why is it a nogo for GiST?

Re: [HACKERS] json accessors

2012-12-05 Thread David E. Wheeler
On Dec 5, 2012, at 10:04 AM, Andrew Dunstan wrote: > Indexing tree-like data isn't at all easy. We don't index XML either. There > has been discussion of this sort of indexing it in the past, and a couple of > people have said they would work on it, but I have not seen a proposal or a > single

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:22 AM, Dimitri Fontaine wrote: > This patch is all about enabling users to create extension without > having to ship them as root on the file system of the database(s) > server(s) first. Right, but it changes the way that existing extensions *dump*, which seems to me to u

Re: [HACKERS] json accessors

2012-12-05 Thread David E. Wheeler
On Dec 5, 2012, at 9:57 AM, Merlin Moncure wrote: > Indexing large documents for fancy querying is a niche case but also > quite complex. This isn't very well covered by xmlpath either btw -- > I think for inspiration we should be looking at hstore. Agreed, although hstore, IIRC, does not suppo

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Josh Berkus
Simon, > And anything which assumes the *absence* of a manual script is also > leaving out a large class of users. ORMs are very important, but not > the only thing we serve. Yes. In the long run, we'll probably need two solutions. An interactive EXPLAIN, and something which logs or aborts for

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 13:18:16 -0500, Tom Lane wrote: > Dimitri Fontaine writes: > > At the SQL level, extensions do live in a database. The only reason why > > we currently have them on the file system is binary executables (.so, > > .dylib, .dll). And those are not per database, not even per cluster, no

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 20:23:29 +0200, Heikki Linnakangas wrote: > On 05.12.2012 20:13, Andres Freund wrote: > >But I really really would like them to go to a per-database directory > >not a per-cluster one. Otherwise the coordination between different > >database "owners" inside a cluster will get really h

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 20:15:42 +0200, Heikki Linnakangas wrote: > On 05.12.2012 20:07, Tom Lane wrote: > >Heikki Linnakangas writes: > >>And whether extension control files (or the same information stored in a > >>table or wherever) should be per-database or per cluster - that's *yet* > >>another separate

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 20:13, Andres Freund wrote: But I really really would like them to go to a per-database directory not a per-cluster one. Otherwise the coordination between different database "owners" inside a cluster will get really hairy. I want to be able to install different versions of an appli

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: > At the SQL level, extensions do live in a database. The only reason why > we currently have them on the file system is binary executables (.so, > .dylib, .dll). And those are not per database, not even per cluster, not > even per major version, they are *per server*. It'

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 20:07, Tom Lane wrote: Heikki Linnakangas writes: And whether extension control files (or the same information stored in a table or wherever) should be per-database or per cluster - that's *yet* another separate issue. You could argue for either behavior. I think anyone arguing

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 12:55:42 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote: > >> And I still don't understand why pg_dump needs to know about any of this... > > > Extensions should be fully per-database and we want pg_dump backups to > > be res

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Tom Lane
Simon Riggs writes: > On 5 December 2012 15:01, Tom Lane wrote: >> And perhaps more to the point, what's the advantage compared to >> good old "log_statement = ddl"? > That logs DDL statements for the whole system and isn't user settable. Not true; you can set it with ALTER ROLE/DATABASE, and a

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Heikki Linnakangas writes: > And whether extension control files (or the same information stored in a > table or wherever) should be per-database or per cluster - that's *yet* > another separate issue. You could argue for either behavior. I think anyone arguing for the former is confusing an in

Re: [HACKERS] json accessors

2012-12-05 Thread Andrew Dunstan
On 12/05/2012 12:29 PM, David E. Wheeler wrote: On Dec 5, 2012, at 9:21 AM, Andrew Dunstan wrote: For sufficiently large columns, I expect I would want a GIN index to speed JSON value extraction queries. Possible with this proposal? Probably not. That greatly reduces its utility for queryi

Re: [HACKERS] json accessors

2012-12-05 Thread Merlin Moncure
On Wed, Dec 5, 2012 at 11:14 AM, David E. Wheeler wrote: > On Nov 28, 2012, at 4:10 PM, Merlin Moncure wrote: > >>> Yes, it's iterative. And for deeply nested json it might be somewhat >>> inefficient, although the parser is pretty fast AFAICT. But it's a start. >> >> not completely buying that:

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Andres Freund writes: > On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote: >> And I still don't understand why pg_dump needs to know about any of this... > Extensions should be fully per-database and we want pg_dump backups to > be restorable into another database/clusters/servers. Wait a m

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Heikki Linnakangas writes: > Ok - but that it yet another issue, not to be confused with how you deploy > extensions. If we are to have such a mode in pg_dump, it should be able to > dump *all* extensions, regardless of how they were deployed. (ok, might be > difficult for extensions that include

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 19:40:58 +0200, Heikki Linnakangas wrote: > On 05.12.2012 19:27, Andres Freund wrote: > >>And I still don't understand why pg_dump needs to know about any of this... > > > >Extensions should be fully per-database and we want pg_dump backups to > >be restorable into another database/cl

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-05 Thread Simon Riggs
On 5 December 2012 15:01, Tom Lane wrote: > John R Pierce writes: >> On 12/5/2012 1:42 AM, Simon Riggs wrote: >>> I think we need a parameter called >>> >>> schema_change_reporting = off (default) | on [USERSET] >>> >>> which displays relevant statistics/reports about the actions taken by >>> D

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 19:27, Andres Freund wrote: And I still don't understand why pg_dump needs to know about any of this... Extensions should be fully per-database and we want pg_dump backups to be restorable into another database/clusters/servers. So having a mode for pg_dump that actually makes dum

Re: [HACKERS] json accessors

2012-12-05 Thread David E. Wheeler
On Dec 5, 2012, at 9:21 AM, Andrew Dunstan wrote: >> For sufficiently large columns, I expect I would want a GIN index to speed >> JSON value extraction queries. Possible with this proposal? > > Probably not. That greatly reduces its utility for querying, though not, of course, for using it i

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote: > On 05.12.2012 12:22, Dimitri Fontaine wrote: > >Heikki Linnakangas writes: > >>No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are > >>very common. But what does that have to do with this patch? > > > >This patch is

Re: [HACKERS] json accessors

2012-12-05 Thread Andrew Dunstan
On 12/05/2012 12:14 PM, David E. Wheeler wrote: On Nov 28, 2012, at 4:10 PM, Merlin Moncure wrote: Yes, it's iterative. And for deeply nested json it might be somewhat inefficient, although the parser is pretty fast AFAICT. But it's a start. not completely buying that: see comments below. n

Re: [HACKERS] json accessors

2012-12-05 Thread Andrew Dunstan
On 12/05/2012 12:11 PM, David E. Wheeler wrote: On Dec 4, 2012, at 10:05 AM, Josh Berkus wrote: json_get(json, variadic text) => json Given that I already do the equivalent in Python, this would suit me well. Not sure about other users ... Well, given that sometimes you will have mixed

  1   2   >