Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Fabien COELHO
Starting to poke at this... the proposal to add prove checks for psql just to see whether \if respects ON_ERROR_STOP seems like an incredibly expensive way to test a rather minor point. On my machine, "make check" in bin/psql goes from zero time to close to 8 seconds. I'm not really on board

Re: [HACKERS] scram and \password

2017-03-11 Thread Michael Paquier
On Sun, Mar 12, 2017 at 9:10 AM, Joe Conway wrote: > On 03/11/2017 03:15 PM, Michael Paquier wrote: >> Yeah but it can be called as well while the application is calling >> PQgetResult() and still looping until it gets a NULL result. Not sure >> if this is a use-case to worry about, but sending a

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread David G. Johnston
On Sat, Mar 11, 2017 at 5:45 PM, Tom Lane wrote: > > * Whether or not you think it's important not to expand skipped variables, > I think that it's critical that skipped backtick expressions not be > executed. > ​ [...] ​ > I do not think that a skipped \if or \elif > should evaluate its argument

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-11 Thread Tomas Vondra
Hi, On 03/07/2017 04:01 AM, Andres Freund wrote: Hi, Attached is an updated version of the patchset, but more importantly some benchmark numbers. I wanted to do a bit of testing and benchmarking on this, but 0004 seems to be a bit broken. The patch does not apply anymore - there are some c

Re: [HACKERS] Parallel Append implementation

2017-03-11 Thread Robert Haas
On Fri, Mar 10, 2017 at 6:01 AM, Tels wrote: > Just a question for me to understand the implementation details vs. the > strategy: > > Have you considered how the scheduling decision might impact performance > due to "inter-plan parallelism vs. in-plan parallelism"? > > So what would be the schedu

Re: [HACKERS] Parallel Append implementation

2017-03-11 Thread Robert Haas
On Fri, Mar 10, 2017 at 8:12 AM, Amit Khandekar wrote: >> +static inline void >> +exec_append_scan_first(AppendState *appendstate) >> +{ >> +appendstate->as_whichplan = 0; >> +} >> >> I don't think this is buying you anything, and suggest backing it out. > > This is required for sequential App

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Robert Haas
On Sat, Mar 11, 2017 at 9:40 PM, Tom Lane wrote: > I thought the same of the version you were complaining about, but > the current patch seems to have dialed it back a good deal. Do you > still find the current error messages unmaintainable? I haven't looked, but I had the impression this had be

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Tom Lane
Andres Freund writes: > On 2017-03-11 11:48:31 -0800, Andres Freund wrote: >> I think that'd be a good plan. We probably should also keep --outputdir >> seperate (which test_decoding/Makefile does, but >> pg_isolation_regress_check doesn't)? > Here's a patch doing that (based on yours). I'd be

Re: [HACKERS] src/test/regress's check-prepared-txns target

2017-03-11 Thread Andrew Dunstan
On 03/11/2017 06:00 PM, Andres Freund wrote: > Hi, > > In > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=ae55d9fbe3871a5e6309d9b91629f1b0ff2b8cba > src/test/regress grew a check-prepared-txns (and an accompanying > installcheck-prepared-txns) target. > > Is that still sensible

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-03-11 Thread Robert Haas
On Fri, Mar 10, 2017 at 7:39 PM, Amit Kapila wrote: > I agree that more analysis can help us to decide if we can use subxids > from PGPROC and if so under what conditions. Have you considered the > another patch I have posted to fix the issue which is to do this > optimization only when subxids a

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Tom Lane
Robert Haas writes: > I think that I have not taken a firm position on what the behavior > should be with respect to errors.I took the position that the > messages being printed saying what happened were too detailed, because > they not only described what had happened but also tried to > prog

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-11 Thread Mengxing Liu
> -Original Messages- > From: "Kevin Grittner" > Sent Time: 2017-03-12 04:24:29 (Sunday) > To: "Mengxing Liu" > Cc: "pgsql-hackers@postgresql.org" > Subject: Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in > serializable transactions > > On Fri, Mar 10, 2017 at 9

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-11 Thread Robert Haas
On Sat, Mar 11, 2017 at 12:20 AM, Amit Kapila wrote: >> /* >> + * Change the shared buffer state in critical section, >> + * otherwise any error could make it unrecoverable after >> + * recovery. >> + */ >> +

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-11 Thread Robert Haas
On Fri, Mar 10, 2017 at 7:42 PM, Bruce Momjian wrote: > On Fri, Mar 10, 2017 at 07:15:59PM -0500, Peter Eisentraut wrote: >> On 3/10/17 14:40, Andres Freund wrote: >> > I'd really like to get it in. The performance improvements on its own >> > are significant, and it provides the basis for signif

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Robert Haas
On Fri, Mar 3, 2017 at 3:18 AM, Fabien COELHO wrote: > I'm ok with this patch. I think that the very simple automaton code > structure achieved is worth the very few code duplications. It is also > significantly shorter than the nested if/switch variants, and it does > exactly what Tom and Robert

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Tom Lane
Corey Huinker writes: > [ 0001.if_endif.v21.diff ] I had thought that this patch was pretty close to committable, but the more I poke at it the less I think so. The technology it uses for skipping unexecuted script sections has got too many bugs. * Daniel Verite previously pointed out the desir

Re: [HACKERS] scram and \password

2017-03-11 Thread Joe Conway
On 03/11/2017 03:15 PM, Michael Paquier wrote: > Yeah but it can be called as well while the application is calling > PQgetResult() and still looping until it gets a NULL result. Not sure > if this is a use-case to worry about, but sending a query to the > server in PQencryptPassword() could as wel

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-11 Thread Noah Misch
On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote: > Today I played with xml_recv function and with xml processing functions. > > xml_recv function ensures correct encoding from document encoding to server > encoding. But the decl section holds original encoding info - that should > be

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Corey Huinker
On Sat, Mar 11, 2017 at 4:17 PM, Tom Lane wrote: > Corey Huinker writes: > > [ 0001.if_endif.v21.diff ] > > Starting to poke at this... the proposal to add prove checks for psql > just to see whether \if respects ON_ERROR_STOP seems like an incredibly > expensive way to test a rather minor point

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Andres Freund
On 2017-03-11 11:48:31 -0800, Andres Freund wrote: > On 2017-03-11 12:05:23 -0500, Tom Lane wrote: > > I wrote: > > > I believe the core problem is that contrib/test_decoding's regresscheck > > > and isolationcheck targets each want to use ./tmp_check as their > > > --temp-instance. make has no re

Re: [HACKERS] scram and \password

2017-03-11 Thread Michael Paquier
On Sun, Mar 12, 2017 at 8:04 AM, Joe Conway wrote: > On 03/11/2017 02:21 PM, Michael Paquier wrote: >> On Sun, Mar 12, 2017 at 5:35 AM, Joe Conway wrote: >>> So if the password is not already set, \password uses >>> password_encryption to determine which format to use, and if the >>> password is

Re: [HACKERS] scram and \password

2017-03-11 Thread Joe Conway
On 03/11/2017 02:21 PM, Michael Paquier wrote: > On Sun, Mar 12, 2017 at 5:35 AM, Joe Conway wrote: >> So if the password is not already set, \password uses >> password_encryption to determine which format to use, and if the >> password is already set, then the current method is assumed. > > Yeah

[HACKERS] src/test/regress's check-prepared-txns target

2017-03-11 Thread Andres Freund
Hi, In https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=ae55d9fbe3871a5e6309d9b91629f1b0ff2b8cba src/test/regress grew a check-prepared-txns (and an accompanying installcheck-prepared-txns) target. Is that still sensible given that pg_regress actually enables prepared transactions?

Re: [HACKERS] Changing references of password encryption to hashing

2017-03-11 Thread Joe Conway
On 03/09/2017 06:16 PM, Michael Paquier wrote: > As discussed here: > https://www.postgresql.org/message-id/98cafcd0-5557-0bdf-4837-0f2b7782d...@joeconway.com > We are using in documentation and code comments "encryption" to define > what actually is hashing, which is confusing. > > Attached is a

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Andres Freund
On 2017-03-11 14:17:42 -0800, Andres Freund wrote: > On 2017-03-07 09:36:51 -0300, Alvaro Herrera wrote: > > FWIW, +1 on improving matters here. > > > > Andres Freund wrote: > > > > > The best I can come up so far is a toplevel target that creates the temp > > > install, starts a cluster and then r

Re: [HACKERS] scram and \password

2017-03-11 Thread Michael Paquier
On Sun, Mar 12, 2017 at 5:35 AM, Joe Conway wrote: > On 03/10/2017 02:43 PM, Michael Paquier wrote: >> Instead of changing the default, I think that we should take this >> occasion to rename PQencryptPassword to something like >> PQhashPassword(), and extend it with a method argument to support bo

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Andres Freund
On 2017-03-07 09:36:51 -0300, Alvaro Herrera wrote: > FWIW, +1 on improving matters here. > > Andres Freund wrote: > > > The best I can come up so far is a toplevel target that creates the temp > > install, starts a cluster and then runs the 'installcheck-or-check' > > target on all the subdirector

[HACKERS] Preserving param location

2017-03-11 Thread Julien Rouhaud
Hello, When a query contains parameters, the original param node contains the token location. However, this information is lost when the Const node is generated, this one will only contain position -1 (unknown). FWIW, we do have a use case for this (custom extension that tracks quals statistics,

Re: [HACKERS] How to get the 'ctid' from a record type?

2017-03-11 Thread Eric Ridge
On Sat, Mar 11, 2017 at 2:14 PM Andres Freund wrote: > On 2017-03-11 04:31:16 +, Eric Ridge wrote: > > Well shoot. That kinda spoils my plans. > > I think you should elaborate on what you're trying to achieve - > otherwise our advice will be affected by the recent, widely reported, > crystal

Re: [HACKERS] [bug fix] dblink leaks unnamed connections

2017-03-11 Thread Joe Conway
On 03/09/2017 08:31 AM, Joe Conway wrote: > On 03/09/2017 07:54 AM, Tom Lane wrote: >> Fujii Masao writes: >>> On Wed, Mar 8, 2017 at 3:52 PM, Tsunakawa, Takayuki >>> wrote: dblink fails to close the unnamed connection as follows when a new unnamed connection is requested. The attache

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-11 Thread Tom Lane
Corey Huinker writes: > [ 0001.if_endif.v21.diff ] Starting to poke at this... the proposal to add prove checks for psql just to see whether \if respects ON_ERROR_STOP seems like an incredibly expensive way to test a rather minor point. On my machine, "make check" in bin/psql goes from zero time

Re: [HACKERS] How to get the 'ctid' from a record type?

2017-03-11 Thread Andres Freund
On 2017-03-11 04:31:16 +, Eric Ridge wrote: > Well shoot. That kinda spoils my plans. I think you should elaborate on what you're trying to achieve - otherwise our advice will be affected by the recent, widely reported, crystal ball scarcity. - Andres -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] INSERT INTO arr2(array[1].d, array[2].d)

2017-03-11 Thread Andres Freund
On 2017-03-11 14:43:55 -0600, Jim Nasby wrote: > Over in [1], I was very surprised to discover $SUBJECT[2]. I looked in the > docs, and they clearly indicate that INSERT accepts "column names". They also say "The column name can be qualified with a subfield name or array subscript, if needed." >

Re: [HACKERS] How to get the 'ctid' from a record type?

2017-03-11 Thread Jim Nasby
On 3/10/17 10:31 PM, Eric Ridge wrote: What about this? Is the tuple currently being evaluated (I suppose in the case of a sequential scan) available in the context of a function call? AFAIK that *very* specific case would work, because the executor would be handing you the raw tuple. Not a g

[HACKERS] INSERT INTO arr2(array[1].d, array[2].d)

2017-03-11 Thread Jim Nasby
Over in [1], I was very surprised to discover $SUBJECT[2]. I looked in the docs, and they clearly indicate that INSERT accepts "column names". What's the best way to describe this? "column expression"? "field expression"? 1: https://www.postgresql.org/message-id/20170311005810.kuccp7t5t5jhe.

Re: [HACKERS] scram and \password

2017-03-11 Thread Joe Conway
On 03/10/2017 02:43 PM, Michael Paquier wrote: > On Sat, Mar 11, 2017 at 2:53 AM, Jeff Janes wrote: >> Should the \password tool in psql inspect password_encryption and act on it >> being 'scram'? > > Not sure if it is wise to change the default fot this release. > >> I didn't see this issue dis

Re: [HACKERS] Index usage for elem-contained-by-const-range clauses

2017-03-11 Thread Jim Nasby
On 3/10/17 8:29 AM, Alexander Korotkov wrote: That's cool idea. But I would say more. Sometimes it's useful to transform "intcol between x and y" into "intcol <@ 'x,y'::int4range". btree_gin supports "intcol between x and y" as overlap of "intcol >= x" and "intcol <= y". That is very ineffici

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-11 Thread Kevin Grittner
On Fri, Mar 10, 2017 at 9:12 PM, Mengxing Liu wrote: > My name is Mengxing Liu. I am interested in the project "Eliminate > O(N^2) scaling from rw-conflict tracking in serializable > transactions”. After discussing with Kevin off-list, I think it's > time to post discussion here. I am afraid that

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Jim Nasby
On 3/11/17 2:06 PM, Tom Lane wrote: Jim Nasby writes: It's actually a lot harder to mess up providing a git repo link than manually submitting patches to the mailing list. Yeah, we've heard that proposal before. We're still not doing it though. Insisting on patches being actually submitted to

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Tom Lane
Jim Nasby writes: > It's actually a lot harder to mess up providing a git repo link than > manually submitting patches to the mailing list. Yeah, we've heard that proposal before. We're still not doing it though. Insisting on patches being actually submitted to the mailing list is important for

Re: [HACKERS] Explicit subtransactions for PL/Tcl

2017-03-11 Thread Tom Lane
Pavel Stehule writes: > I'll mark this patch as ready for commiter I've pushed this after mostly-cosmetic cleanup. One thing I changed that's not cosmetic is to put MemoryContextSwitchTo(oldcontext); after the BeginInternalSubTransaction call. I see there was some discussion upthread

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Andres Freund
On 2017-03-11 12:05:23 -0500, Tom Lane wrote: > I wrote: > > I believe the core problem is that contrib/test_decoding's regresscheck > > and isolationcheck targets each want to use ./tmp_check as their > > --temp-instance. make has no reason to believe it shouldn't run those > > two sub-jobs in pa

[HACKERS] proposal - Default namespaces for XPath expressions (PostgreSQL 11)

2017-03-11 Thread Pavel Stehule
Hi This proposal is followup of implementation of XMLTABLE. Lot of XML documents has assigned document namespace. http://x.y";>10 For these XML document any search path must use schema "http://x.y";. This is not too intuitive, and from XMLTABLE usage is not too user friendly, because the defaul

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Jim Nasby
On 3/10/17 6:06 PM, Peter Eisentraut wrote: On 3/10/17 19:00, Jim Nasby wrote: Maybe instead of having the commitfest app try and divine patches from the list it should be able to send patches to the list from a specified git repo/branch. Anyone that provides that info would have tests run autom

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-03-11 Thread Oleg Bartunov
On Fri, Mar 10, 2017 at 7:07 AM, Petr Jelinek wrote: > On 09/03/17 19:50, Peter van Hardenberg wrote: > > Anecdotally, we just stored dates as strings and used a convention (key > > ends in "_at", I believe) to interpret them. The lack of support for > > dates in JSON is well-known, universally d

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-11 Thread Tom Lane
I wrote: > I believe the core problem is that contrib/test_decoding's regresscheck > and isolationcheck targets each want to use ./tmp_check as their > --temp-instance. make has no reason to believe it shouldn't run those > two sub-jobs in parallel, but when it does, we get two postmasters trying

Re: [HACKERS] make check-world output

2017-03-11 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> What about just reverting 2f227656076a? > That works for me too, if we think we no longer need that level of > detail. A general issue with this sort of messaging is that when things are working more or less normally, you'd just as soon not see it ...

Re: [HACKERS] background sessions

2017-03-11 Thread Pavel Stehule
2017-03-09 14:52 GMT+01:00 Peter Eisentraut : > On 3/8/17 14:22, Pavel Stehule wrote: > > 1. will be background session process closed automatically when parent > > process is closed? > > If the communications queue goes away the process will eventually die. > This is similar to how a backend proc

Re: [HACKERS] make check-world output

2017-03-11 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Jeff Janes wrote: > >> There was some recent discussion about making "make check-world" faster. > >> I'm all for that, but how about making it quieter? On both machines I've > >> run it on (CentOS6.8 and Ubuntu 16.04.2), it dumps some gibberish to > >>

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-03-11 Thread Pavel Stehule
2017-03-10 15:45 GMT+01:00 Alexander Korotkov : > On Fri, Mar 10, 2017 at 5:10 PM, Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > >> On 2/24/17 16:32, Pavel Stehule wrote: >> > set EXTENDED_DESCRIBE_SORT size_desc >> > \dt+ >> > \l+ >> > \di+ >> > >> > Possible

Re: [HACKERS] Explicit subtransactions for PL/Tcl

2017-03-11 Thread Pavel Stehule
2017-03-10 20:31 GMT+01:00 Victor Wagner : > On Thu, 9 Mar 2017 12:04:31 +0100 > Pavel Stehule wrote: > > > > > Now test demonstrate how errors uncaught on the Tcl level interact > > > with postgresql error system. > > > > > > > you can catch the exception outside and write own message > > OK, he

Re: [HACKERS] Speedup twophase transactions

2017-03-11 Thread Nikhil Sontakke
Hi David and Michael, > It would be great to get this thread closed out after 14 months and many > commits. > > PFA, latest patch which addresses Michael's comments. twophase.c: In function ‘PrepareRedoAdd’: > twophase.c:2539:20: warning: variable ‘gxact’ set but not used > [-Wunused-but-set-var