[HACKERS] Re: proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-07-24 Thread Pavel Stehule
Hello 2013/7/25 Stephen Frost : > All, > > * Rushabh Lathia (rushabh.lat...@gmail.com) wrote: >> Latest patch looks good to me. > > I've committed this, with some pretty serious clean-up. In the future, > please don't simply copy/paste code w/o any updates to the comments > included, particularly

Re: [HACKERS] dynamic background workers, round two

2013-07-24 Thread Michael Paquier
On Thu, Jul 25, 2013 at 8:05 AM, Andres Freund wrote: > On 2013-07-25 08:03:17 +0900, Michael Paquier wrote: > > On Thu, Jul 25, 2013 at 6:34 AM, Andres Freund >wrote: > > > > > > --- a/contrib/worker_spi/worker_spi.c > > > > +++ b/contrib/worker_spi/worker_spi.c > > > > > > Btw, I've posted a mi

Re: [HACKERS] Preventing tuple-table leakage in plpgsql

2013-07-24 Thread Tom Lane
I wrote: > Hmm ... good point. The other plan I'd been considering was to add > explicit tracking inside spi.c of all tuple tables created within the > current procedure, and then have AtEOSubXact_SPI flush any that were > created inside a failed subxact. The tables would still be children of > t

[HACKERS] Re: proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-07-24 Thread Stephen Frost
All, * Rushabh Lathia (rushabh.lat...@gmail.com) wrote: > Latest patch looks good to me. I've committed this, with some pretty serious clean-up. In the future, please don't simply copy/paste code w/o any updates to the comments included, particularly when the comments no longer make sense in the

Re: [HACKERS] dynamic background workers, round two

2013-07-24 Thread Andres Freund
On 2013-07-25 08:03:17 +0900, Michael Paquier wrote: > On Thu, Jul 25, 2013 at 6:34 AM, Andres Freund wrote: > > > > --- a/contrib/worker_spi/worker_spi.c > > > +++ b/contrib/worker_spi/worker_spi.c > > > > Btw, I've posted a minimal regression test for bworkers/worker_spi in > > 20130724175742.gd

Re: [HACKERS] dynamic background workers, round two

2013-07-24 Thread Michael Paquier
On Thu, Jul 25, 2013 at 6:34 AM, Andres Freund wrote: > > --- a/contrib/worker_spi/worker_spi.c > > +++ b/contrib/worker_spi/worker_spi.c > > Btw, I've posted a minimal regression test for bworkers/worker_spi in > 20130724175742.gd10...@alap2.anarazel.de - I'd like to see some coverage > of this..

Re: [HACKERS] dynamic background workers, round two

2013-07-24 Thread Michael Paquier
On Thu, Jul 25, 2013 at 6:34 AM, Andres Freund wrote: > Hi, > > On 2013-07-24 12:46:21 -0400, Robert Haas wrote: > > The attached patch attempts to remedy this problem. When you register > > a background worker, you can obtain a "handle" that can subsequently > > be used to query for the worker's

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Tom Lane
Andres Freund writes: > The first attached patch adds slist_delete_current(), updates the > comments addressing your points and converts the bgworker code to pass > the iterator around (it's more efficient which might actually matter > with a few hundred bgworkers). Applied with fixes. The slist

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Andrew Gierth
Tom Lane said: > If we did it with a WithOrdinality expression node, the result would > always be of type RECORD, and we'd have to use blessed tuple > descriptors to keep track of exactly which record type a particular > instance emits. This is certainly do-able (see RowExpr for > precedent). May

Re: [HACKERS] dynamic background workers, round two

2013-07-24 Thread Andres Freund
Hi, On 2013-07-24 12:46:21 -0400, Robert Haas wrote: > The attached patch attempts to remedy this problem. When you register > a background worker, you can obtain a "handle" that can subsequently > be used to query for the worker's PID. If you additionally initialize > bgw_notify_pid = MyProcPid

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Andres Freund
> Andres Freund writes: > > The first attached patch adds slist_delete_current(), updates the > > comments addressing your points and converts the bgworker code to pass > > the iterator around (it's more efficient which might actually matter > > with a few hundred bgworkers). > > I found the added

Re: [HACKERS] [GENERAL] Insert result does not match record count

2013-07-24 Thread Tom Lane
Andres Freund writes: > I think fixing this for 9.4 is fine, but due to the compat issues I > think it's to late for 9.3. Agreed -- this is effectively a protocol change, albeit a pretty minor one, so I can't see back-patching it. regards, tom lane -- Sent via pgsql-ha

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Tom Lane
Andres Freund writes: > The first attached patch adds slist_delete_current(), updates the > comments addressing your points and converts the bgworker code to pass > the iterator around (it's more efficient which might actually matter > with a few hundred bgworkers). > I found the added newlines in

Re: [HACKERS] [GENERAL] Insert result does not match record count

2013-07-24 Thread Andres Freund
On 2013-07-24 13:48:23 -0400, Tom Lane wrote: > Vik Fearing writes: > > Also worth mentioning is bug #7766. > > http://www.postgresql.org/message-id/e1tlli5-0007tr...@wrigleys.postgresql.org > > Yeah, did you read that whole thread? The real issue here is going to > be whether client-side code f

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Andres Freund
On 2013-07-24 13:36:39 -0400, Tom Lane wrote: > Robert Haas writes: > > On Tue, Jul 23, 2013 at 9:38 PM, Tom Lane wrote: > >> If it weren't that we've been speculating for years about deprecating > >> SRFs-in-tlists once we had LATERAL, I would personally consider this > >> patch DOA in this form

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Tom Lane
Greg Stark writes: > On Wed, Jul 24, 2013 at 6:39 PM, Robert Haas wrote: >> This patch will introduce, without documentation, a fifth class of >> keyword. ORDINALITY will need to be quoted when, and only when, it >> immediately follows WITH. Without some change to our deparsing code, >> this is

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Andres Freund
Hi, On 2013-07-24 11:49:20 -0400, Tom Lane wrote: > Andres Freund writes: > > This will require another member variable in slist_mutable_iter which > > obviously will need to be maintained, but that seems fine to me since it > > will reduce the cost of actually deleting noticeably. > > I think t

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Greg Stark
On Wed, Jul 24, 2013 at 6:39 PM, Robert Haas wrote: > This patch will introduce, without documentation, a fifth class of > keyword. ORDINALITY will need to be quoted when, and only when, it > immediately follows WITH. Without some change to our deparsing code, > this is a dump/restore hazard; an

Re: [HACKERS] Adding Zigzag Merge Join to Index Nested Loops Join

2013-07-24 Thread Robert Haas
On Tue, Jul 23, 2013 at 3:40 PM, tubadzin wrote: > I want add Zigzag Merge join to Index Nested Loops Join alghoritm. > http://www.cs.berkeley.edu/~fox/summaries/database/query_eval_5-8.html > Which files are responsible for Index nested loops join ? (not simple nested > loops join which has doubl

Re: [HACKERS] Patch to add support of "IF NOT EXISTS" to others "CREATE" statements

2013-07-24 Thread Karol Trzcionka
Hello, patch works fine but is there any reason to comparing each ifNotExists in different way? i.e. ProcedureCreate if (!ifNotExists) ... else { ... return } TypeCreate if (ifNotExists) { ... return } ... --- Shouldn't it be more consistent? Regards, Karol -- Sent via pgsql-hackers mailing

Re: [HACKERS] [GENERAL] Insert result does not match record count

2013-07-24 Thread Tom Lane
Vik Fearing writes: > Also worth mentioning is bug #7766. > http://www.postgresql.org/message-id/e1tlli5-0007tr...@wrigleys.postgresql.org Yeah, did you read that whole thread? The real issue here is going to be whether client-side code falls over on wider-than-32-bit counts. We can fix the back

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Greg Stark
On Wed, Jul 24, 2013 at 6:36 PM, Tom Lane wrote: > That seems to me to be unlikely to happen, because it would be > impossible to preserve the current (admittedly bad) semantics. > If we're going to change the behavior at all we might as well just > drop the feature, IMO. It would be nice to supp

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Robert Haas
On Wed, Jul 24, 2013 at 1:36 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jul 23, 2013 at 9:38 PM, Tom Lane wrote: >>> If it weren't that we've been speculating for years about deprecating >>> SRFs-in-tlists once we had LATERAL, I would personally consider this >>> patch DOA in this form

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Robert Haas
On Fri, Jul 19, 2013 at 1:50 PM, Greg Stark wrote: > My only reservation with this patch is whether the WITH_ORDINALITY > parser hack is the way we want to go. The precedent was already set > with WITH TIME ZONE though and I think this was the best option. I share this reservation. Lexer hacks a

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 23, 2013 at 9:38 PM, Tom Lane wrote: >> If it weren't that we've been speculating for years about deprecating >> SRFs-in-tlists once we had LATERAL, I would personally consider this >> patch DOA in this form. > I guess I'd sort of assumed that the plan was to co

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-07-24 Thread Robert Haas
On Tue, Jul 23, 2013 at 9:38 PM, Tom Lane wrote: > If it weren't that we've been speculating for years about deprecating > SRFs-in-tlists once we had LATERAL, I would personally consider this > patch DOA in this form. If we do think we'll probably deprecate that > feature, then not extending WITH

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-24 Thread Robert Haas
On Tue, Jul 23, 2013 at 12:13 PM, Greg Smith wrote: > On 7/23/13 10:56 AM, Robert Haas wrote: >> On Mon, Jul 22, 2013 at 11:48 PM, Greg Smith wrote: >>> >>> We know that a 1GB relation segment can take a really long time to write >>> out. That could include up to 128 changed 8K pages, and we all

[HACKERS] dynamic background workers, round two

2013-07-24 Thread Robert Haas
The dynamic background workers patch that I submitted for CF1 was generally well-received, but several people commented on a significant limitation: there's currently no way for a backend that requests a new background worker to know whether that background worker was successfully started. If you'

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-07-24 11:53:39 -0400, Alvaro Herrera wrote: > > Andres Freund wrote: > > Amusingly, this will mean ForgetBackgroundWorker() will need to have a > > slist_mutable_iter * argument if it wants to use the new function ... > > The only alternative I see would be to pass

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Andres Freund
On 2013-07-24 11:53:39 -0400, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2013-07-24 11:26:00 -0400, Tom Lane wrote: > > > > So I'm going to end up hand-implementing the same kind of manipulation > > > we frequently use with traditional Lists, namely keep a second variable > > > that's the

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-07-24 11:26:00 -0400, Tom Lane wrote: > > So I'm going to end up hand-implementing the same kind of manipulation > > we frequently use with traditional Lists, namely keep a second variable > > that's the preceding list element (not the next one) so I can unlink and >

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Tom Lane
Andres Freund writes: > On 2013-07-24 11:26:00 -0400, Tom Lane wrote: >> So I'm going to end up hand-implementing the same kind of manipulation >> we frequently use with traditional Lists, namely keep a second variable >> that's the preceding list element (not the next one) so I can unlink and >>

Re: [HACKERS] [GENERAL] Insert result does not match record count

2013-07-24 Thread Vik Fearing
On 07/24/2013 04:04 PM, Vik Fearing wrote: > On 07/22/2013 06:20 PM, Jeff Janes wrote: >> On Fri, Jul 19, 2013 at 3:20 PM, Natalie Wenz >> wrote: >>> Hi all, >>> >>> I am moving some data from one table to another in 9.2.4, and keep seeing >>> this strange scenario: >>> >>> insert into newtable

Re: [HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Andres Freund
On 2013-07-24 11:26:00 -0400, Tom Lane wrote: > So I went off to implement the SPITupleTable tracking discussed in > <6553.1374424...@sss.pgh.pa.us>, and thought it would be cool to > use the slist infrastructure defined in lib/ilist.h rather than > creating a separate List node for each SPITupleTa

[HACKERS] ilist.h is not useful as-is

2013-07-24 Thread Tom Lane
So I went off to implement the SPITupleTable tracking discussed in <6553.1374424...@sss.pgh.pa.us>, and thought it would be cool to use the slist infrastructure defined in lib/ilist.h rather than creating a separate List node for each SPITupleTable struct. However, I soon ran into a problem: there'

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Andres Freund
On 2013-07-24 10:57:14 -0400, Tom Lane wrote: > I wrote: > > The only thing here that really bothers me is that a crash during DROP > > DATABASE/TABLESPACE could leave us with a partially populated db/ts > > that's still accessible through the system catalogs. ... > > I guess one thing we could do

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Tom Lane
I wrote: > The only thing here that really bothers me is that a crash during DROP > DATABASE/TABLESPACE could leave us with a partially populated db/ts > that's still accessible through the system catalogs. ... > I guess one thing we could do is create a flag file, say > "dead.dont.use", in the da

Re: [HACKERS] [GENERAL] Insert result does not match record count

2013-07-24 Thread Vik Fearing
On 07/22/2013 06:20 PM, Jeff Janes wrote: > On Fri, Jul 19, 2013 at 3:20 PM, Natalie Wenz wrote: >> Hi all, >> >> I am moving some data from one table to another in 9.2.4, and keep seeing >> this strange scenario: >> >> insert into newtable select data from oldtable where proc_date >= x and >> p

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Andres Freund
On 2013-07-24 16:15:59 +0300, Heikki Linnakangas wrote: > >For DROP DATABASE, without something like incomplete actions, > >piggybacking on the commit record doesn't solve the issue of > >CHECKPOINTS > >either, because the commit record you piggybacked on could have > >committed before a checkpoint

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Tom Lane
Heikki Linnakangas writes: > That's no different from CREATE TABLE / INDEX and DROP TABLE / INDEX. E.g. If > you crash after CREATE TABLE but before COMMIT, the file is leaked. But it's > just a waste of space, everything still works. Well, it is different, because if you crash partway through

Re: [HACKERS] Suggestion for concurrent index creation using a single full scan operation

2013-07-24 Thread Tim Kane
Wow.. thanks guys, really appreciate the detailed analysis. Tim On Wed, Jul 24, 2013 at 4:08 AM, Noah Misch wrote: > On Tue, Jul 23, 2013 at 01:06:26PM +0100, Tim Kane wrote: > > I haven't given this a lot of thought, but it struck me that when > > rebuilding tables (be it for a restore proces

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Heikki Linnakangas
Andres Freund wrote: >On 2013-07-24 15:45:52 +0300, Heikki Linnakangas wrote: >> Andres Freund wrote: >> >On 2013-07-24 12:59:43 +0200, Andres Freund wrote: >> >> > >> >> What we imo could do would be to drop the tablespaces in a >*separate* >> >> transaction *after* the transaction that removed

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Andres Freund
On 2013-07-24 15:45:52 +0300, Heikki Linnakangas wrote: > Andres Freund wrote: > >On 2013-07-24 12:59:43 +0200, Andres Freund wrote: > >> > > >> What we imo could do would be to drop the tablespaces in a *separate* > >> transaction *after* the transaction that removed the pg_tablespace > >> entry

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Heikki Linnakangas
Andres Freund wrote: >On 2013-07-24 12:59:43 +0200, Andres Freund wrote: >> > >> > Like the DROP TABLE/INDEX case, piggyback the directory deletion >record on >> > the transaction commit record, and eliminate the directory deletion >record >> > altogether. >> >> I don't think burdening commit re

[HACKERS] DATE type output does not follow datestyle parameter

2013-07-24 Thread MauMau
Hello, The description of datestyle parameter does not seem to match the actual behavior. Is this a bug to be fixed? Which do you think should be corrected, the program or the manual? The manual says: DateStyle (string) Sets the display format for date and time values, as well as the rule

Re: [HACKERS] Backup throttling

2013-07-24 Thread Fujii Masao
On Wed, Jul 24, 2013 at 4:20 PM, Antonin Houska wrote: > Hello, > the purpose of this patch is to limit impact of pg_backup on running server. > Feedback is appreciated. Interesting. Please add this patch to the next commitfeat. https://commitfest.postgresql.org/action/commitfest_view?id=19 Rega

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Andres Freund
On 2013-07-24 12:59:43 +0200, Andres Freund wrote: > > > > Like the DROP TABLE/INDEX case, piggyback the directory deletion record on > > the transaction commit record, and eliminate the directory deletion record > > altogether. > > I don't think burdening commit records with that makes sense. It

Re: [HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread Andres Freund
On 2013-07-24 19:30:09 +0900, MauMau wrote: > I've encountered a bug of PITR that corrupts the database. I'm willing to > submit the patch to fix it, but I'm wondering what approach is appropriate. > Could you give me your opinions? > > [Problem] > I cannot connect to the database after performin

[HACKERS] [bug fix] PITR corrupts the database cluster

2013-07-24 Thread MauMau
Hello, I've encountered a bug of PITR that corrupts the database. I'm willing to submit the patch to fix it, but I'm wondering what approach is appropriate. Could you give me your opinions? [Problem] I cannot connect to the database after performing the following steps: 1. CREATE DATABASE m

Re: [HACKERS] Performance problem in PLPgSQL

2013-07-24 Thread Amit Kapila
On Wednesday, July 24, 2013 11:38 AM Amit Kapila wrote: > On Wednesday, July 24, 2013 4:23 AM Tom Lane wrote: > > I wrote: > > > Marc Cousin writes: > > >> The example below is of course extremely simplified, and obviously > > not > > >> what we are really doing in the database, but it exhibits th

[HACKERS] Backup throttling

2013-07-24 Thread Antonin Houska
Hello, the purpose of this patch is to limit impact of pg_backup on running server. Feedback is appreciated. // Antonin Houska (Tony) diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index eb0c1d6..3b7ecfd 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml ++