Re: pgsql: Add documentation for the JIT feature.

2018-03-28 Thread Tom Lane
Andres Freund writes: > Add documentation for the JIT feature. guaibausaurus complains that this broke tarball building: make -C postgresql-11devel/doc/src/sgml/ INSTALL make[1]: Entering directory '/home/pgsql/buildfarm/buildroot/HEAD/pgsql.build/postgresql-11devel/doc/src/sgml' /usr/bin/xsltp

pgsql: Allow committing inside cursor loop

2018-03-28 Thread Peter Eisentraut
Allow committing inside cursor loop Previously, committing or aborting inside a cursor loop was prohibited because that would close and remove the cursor. To allow that, automatically convert such cursors to holdable cursors so they survive commits or rollbacks. Portals now have a new state "aut

Re: pgsql: Optimize btree insertions for common case of increasing values

2018-03-28 Thread Peter Geoghegan
On Wed, Mar 28, 2018 at 11:56 AM, Peter Geoghegan wrote: >> Previously, we agreed that P_IGNORE() is required. So I assume no issues >> there. The other tests seem required too for us to conclusively decide to >> use the cached block. > > Actually, you're right that P_INCOMPLETE_SPLIT() is the onl

Re: pgsql: Add documentation for the JIT feature.

2018-03-28 Thread Simon Riggs
On 28 March 2018 at 22:23, Andres Freund wrote: > Add documentation for the JIT feature. Very nice feature and most welcome but we should call it something other than just "JIT" JIT means Just In Time, which could be applied to many concepts and has been in use for many years in a range of conc

pgsql: C comment: fix typo, log -> lag

2018-03-28 Thread Bruce Momjian
C comment: fix typo, log -> lag Reported-by: atorikoshi Discussion: https://postgr.es/m/b61f2ab9-c0e0-d33d-ce3f-42a228025...@lab.ntt.co.jp Author: atorikoshi Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a2894cce544d120199a1a90469073796d055bb60 Modified Files

pgsql: Fix mistakes in the just added JIT docs.

2018-03-28 Thread Andres Freund
Fix mistakes in the just added JIT docs. Reported-By: Lukas Fittl Author: Andres Freund Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a0a08c1d85dae3c332e7d72a56df0636be9c5d0a Modified Files -- doc/src/sgml/config.sgml | 2 +- doc/src/sgml/jit.sgml

pgsql: Add documentation for the JIT feature.

2018-03-28 Thread Andres Freund
Add documentation for the JIT feature. As promised in earlier commits, this adds documentation about the new build options, the new GUCs, about the planner logic when JIT is used, and the benefits of JIT in general. Also adds a more implementation oriented README. I'm sure we're going to want to

pgsql: Add EXPLAIN support for JIT.

2018-03-28 Thread Andres Freund
Add EXPLAIN support for JIT. This just shows a few details about JITing, e.g. how many functions have been JITed, and how long that took. To avoid noise in regression tests with functions sometimes being JITed in --with-llvm builds, disable display when COSTS OFF is specified. Author: Andres Fre

pgsql: Add inlining support to LLVM JIT provider.

2018-03-28 Thread Andres Freund
Add inlining support to LLVM JIT provider. This provides infrastructure to allow JITed code to inline code implemented in C. This e.g. can be postgres internal functions or extension code. This already speeds up long running queries, by allowing the LLVM optimizer to optimize across function boun

pgsql: Use isinf builtin for clang, for performance.

2018-03-28 Thread Andres Freund
Use isinf builtin for clang, for performance. When compiling with clang glibc's definition of isinf() ends up leading to and external libc function call. That's because there was a bug in the builtin in an old gcc version, and clang claims compatibility with an older version. That causes clang to

pgsql: Make pg_rewind skip files and directories that are removed durin

2018-03-28 Thread Fujii Masao
Make pg_rewind skip files and directories that are removed during server start. The target cluster that was rewound needs to perform recovery from the checkpoint created at failover, which leads it to remove or recreate some files and directories that may have been copied from the source cluster.

pgsql: Fix handling of files that source server removes during pg_rewin

2018-03-28 Thread Fujii Masao
Fix handling of files that source server removes during pg_rewind is running. After processing the filemap to build the list of chunks that will be fetched from the source to rewing the target server, it is possible that a file which was previously processed is removed from the source. A simple e

pgsql: Fix handling of files that source server removes during pg_rewin

2018-03-28 Thread Fujii Masao
Fix handling of files that source server removes during pg_rewind is running. After processing the filemap to build the list of chunks that will be fetched from the source to rewing the target server, it is possible that a file which was previously processed is removed from the source. A simple e

pgsql: Fix handling of files that source server removes during pg_rewin

2018-03-28 Thread Fujii Masao
Fix handling of files that source server removes during pg_rewind is running. After processing the filemap to build the list of chunks that will be fetched from the source to rewing the target server, it is possible that a file which was previously processed is removed from the source. A simple e

pgsql: Fix handling of files that source server removes during pg_rewin

2018-03-28 Thread Fujii Masao
Fix handling of files that source server removes during pg_rewind is running. After processing the filemap to build the list of chunks that will be fetched from the source to rewing the target server, it is possible that a file which was previously processed is removed from the source. A simple e

Re: pgsql: Optimize btree insertions for common case of increasing values

2018-03-28 Thread Peter Geoghegan
On Wed, Mar 28, 2018 at 2:14 AM, Pavan Deolasee wrote: >> This code can allocate memory in a critical section, since >> RelationSetTargetBlock() can call smgropen(): > I think you're right. I propose that we call RelationSetTargetBlock right > after the critical section ends. That should not have

pgsql: PL/pgSQL: Nested CALL with transactions

2018-03-28 Thread Peter Eisentraut
PL/pgSQL: Nested CALL with transactions So far, a nested CALL or DO in PL/pgSQL would not establish a context where transaction control statements were allowed. This fixes that by handling CALL and DO specially in PL/pgSQL, passing the atomic/nonatomic execution context through and doing the requ

pgsql: Fix actual and potential double-frees around tuplesort usage.

2018-03-28 Thread Tom Lane
Fix actual and potential double-frees around tuplesort usage. tuplesort_gettupleslot() passed back tuples allocated in the tuplesort's own memory context, even when the caller was responsible to free them. This created a double-free hazard, because some callers might destroy the tuplesort object (

pgsql: Fix actual and potential double-frees around tuplesort usage.

2018-03-28 Thread Tom Lane
Fix actual and potential double-frees around tuplesort usage. tuplesort_gettupleslot() passed back tuples allocated in the tuplesort's own memory context, even when the caller was responsible to free them. This created a double-free hazard, because some callers might destroy the tuplesort object (

pgsql: Fix actual and potential double-frees around tuplesort usage.

2018-03-28 Thread Tom Lane
Fix actual and potential double-frees around tuplesort usage. tuplesort_gettupleslot() passed back tuples allocated in the tuplesort's own memory context, even when the caller was responsible to free them. This created a double-free hazard, because some callers might destroy the tuplesort object (

pgsql: Fix actual and potential double-frees around tuplesort usage.

2018-03-28 Thread Tom Lane
Fix actual and potential double-frees around tuplesort usage. tuplesort_gettupleslot() passed back tuples allocated in the tuplesort's own memory context, even when the caller was responsible to free them. This created a double-free hazard, because some callers might destroy the tuplesort object (

pgsql: Store 2PC GID in commit/abort WAL recs for logical decoding

2018-03-28 Thread Simon Riggs
Store 2PC GID in commit/abort WAL recs for logical decoding Store GID of 2PC in commit/abort WAL records when wal_level = logical. This allows logical decoding to send the SAME gid to subscribers across restarts of logical replication. Track relica origin replay progress for 2PC. (Edited from pa

pgsql: Attempt to fix jsonb_plpython build on Windows

2018-03-28 Thread Peter Eisentraut
Attempt to fix jsonb_plpython build on Windows Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/75e95dd79ba22e18687a069d2ff2fd29afab5798 Modified Files -- src/tools/msvc/Install.pm | 1 + src/tools/msvc/Mkvcbuild.pm | 40 ++--

pgsql: Fix jsonb_plpython tests on older Python versions

2018-03-28 Thread Peter Eisentraut
Fix jsonb_plpython tests on older Python versions Rewrite one test to avoid a case where some Python versions have output format differences (Decimal('1') vs Decimal("1")). Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e81fc9b9dbf9d744dcc9fb210e4353a350be1e22 Mod

Re: pgsql: Transforms for jsonb to PL/Python

2018-03-28 Thread Peter Eisentraut
On 3/28/18 08:39, Peter Eisentraut wrote: > Transforms for jsonb to PL/Python Needs fixes for Windows build. I can give it a blind attempt, but help would be welcome. I'm looking into the regression test failures that appear to be minor output differences. -- Peter Eisentraut http

pgsql: Transforms for jsonb to PL/Python

2018-03-28 Thread Peter Eisentraut
Transforms for jsonb to PL/Python Add a new contrib module jsonb_plpython that provide a transform between jsonb and PL/Python. jsonb values are converted to appropriate Python types such as dicts and lists, and vice versa. Author: Anthony Bykov Reviewed-by: Aleksander Alekseev Reviewed-by: Ni

Re: pgsql: Fast ALTER TABLE ADD COLUMN with a non-NULL default

2018-03-28 Thread Etsuro Fujita
(2018/03/28 13:25), Tom Lane wrote: Andres Freund writes: There's also a few of that annoying postgres_fdw thing. Some of my critters hit that regularly :(. We need to figure out what's going on there. I plan to work on this after the commitfest. Yeah, really. I've made several sustained a

Re: pgsql: Optimize btree insertions for common case of increasing values

2018-03-28 Thread Pavan Deolasee
On Wed, Mar 28, 2018 at 11:33 AM, Peter Geoghegan wrote: > > > This code can allocate memory in a critical section, since > RelationSetTargetBlock() can call smgropen(): > > if (P_ISLEAF(lpageop)) > + { > xlinfo = XLOG_BTREE_INSERT_LEAF; > + > +

Re: pgsql: Allow HOT updates for some expression indexes

2018-03-28 Thread Konstantin Knizhnik
On 28.03.2018 07:23, Simon Riggs wrote: On 27 March 2018 at 22:26, Tom Lane wrote: I wrote: The test script appears to imagine that pgstats outputs update instantaneously. Oh, wait, it's looking at pg_stat_xact, which is the *local*, unsent stats information. So your problem is actually th