Re: Online enabling of checksums

2018-02-28 Thread Andrey Borodin
> 28 февр. 2018 г., в 6:22, Daniel Gustafsson написал(а): > >> Is there any way we could provide this functionality for previous versions >> (9.6,10)? Like implement utility for offline checksum enabling, without >> WAL-logging, surely. > > While outside the scope of the patch in question (s

Re: Contention preventing locking

2018-02-28 Thread Konstantin Knizhnik
On 28.02.2018 16:32, Amit Kapila wrote: On Mon, Feb 26, 2018 at 8:26 PM, Konstantin Knizhnik wrote: On 26.02.2018 17:20, Amit Kapila wrote: Can you please explain, how it can be done easily without extra tuple locks? I have tried to read your patch but due to lack of comments, it is not cle

RE: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-02-28 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > Yes, it should not copy those WAL files. Most of the time they are going > to be meaningless. See this recent thread: > https://www.postgresql.org/message-id/20180126023609.GH17847%40paquier > .xyz > So I would rather go this way instead of hav

Re: Two-phase update of restart_lsn in LogicalConfirmReceivedLocation

2018-02-28 Thread Craig Ringer
On 1 March 2018 at 13:39, Arseny Sher wrote: > Hello, > > In LogicalConfirmReceivedLocation two fields (data.catalog_xmin and > effective_catalog_xmin) of ReplicationSlot structure are used for > advancing xmin of the slot. This allows to avoid hole when tuples might > already have been vacuumed,

Re: [HACKERS] pgbench randomness initialization

2018-02-28 Thread Fabien COELHO
Hello Tom, Fabien COELHO writes: This is a simple patch that does what it says on the tin. I ran into trouble with the pgbench TAP test *even before applying the patch*, but only because I was doing a VPATH build as a user without 'write' on the source tree (001_pgbench_with_server.pl tried t

Re: server crash in nodeAppend.c

2018-02-28 Thread Rajkumar Raghuwanshi
On Wed, Feb 28, 2018 at 9:29 PM, Robert Haas wrote: > Nice test case. I pushed commit > ce1663cdcdbd9bf15c81570277f70571b3727dd3, including your test case, to > fix this. Thanks Robert for fix and commit. I have reverified commit, this is working fine now. Thanks & Regards, Rajkumar Raghuwans

Re: CALL optional in PL/pgSQL

2018-02-28 Thread Pavel Stehule
2018-03-01 5:51 GMT+01:00 Peter Eisentraut : > This seems to be a popular issue when porting from PL/SQL, so I'll throw > it out here for discussion. Apparently, in PL/SQL you can call another > procedure without the CALL keyword. Here is a patch that attempts to > implement that in PL/pgSQL as

Re: Synchronous replay take III

2018-02-28 Thread Thomas Munro
On Thu, Mar 1, 2018 at 2:39 PM, Thomas Munro wrote: > I was pinged off-list by a fellow -hackers denizen interested in the > synchronous replay feature and wanting a rebased patch to test. Here > it goes, just in time for a Commitfest. Please skip to the bottom of > this message for testing note

Re: 2018-03 Commitfest starts tomorrow

2018-02-28 Thread Tom Lane
David Steele writes: > I'll be starting the Commitfest at midnight AoE (07:00 ET, 13:00 CET) so > please get your patches in before then. > Please remember that if you drop a new and large (or invasive patch) > into this CF it may be moved to the next CF. > This last CF for PG11 should generally b

Two-phase update of restart_lsn in LogicalConfirmReceivedLocation

2018-02-28 Thread Arseny Sher
Hello, In LogicalConfirmReceivedLocation two fields (data.catalog_xmin and effective_catalog_xmin) of ReplicationSlot structure are used for advancing xmin of the slot. This allows to avoid hole when tuples might already have been vacuumed, but slot's state was not yet flushed to the disk: if we c

Re: PATCH: Unlogged tables re-initialization tests

2018-02-28 Thread Thomas Munro
On Thu, Mar 1, 2018 at 9:24 AM, David Steele wrote: > These tests were originally included in the exclude unlogged tables > patch [1] to provide coverage for the refactoring of reinit.c. Hi David, +# The following tests test symlinks. Windows doesn't have symlinks, so +# skip on Windows. Could

Re: Why chain of snapshots is used in ReorderBufferCommit?

2018-02-28 Thread Andres Freund
Hi, On 2018-03-01 08:17:33 +0300, Arseny Sher wrote: > While prowling through snapbuild & reorderbuffer code, I wondered: why a queue > of snapshots is used for replaying each transaction instead of just picking up > snapshot from snapbuilder once when COMMIT record is read? I am not aware of > a

Re: Online enabling of checksums

2018-02-28 Thread Andrey Borodin
> 28 февр. 2018 г., в 22:06, Robert Haas написал(а): > > On Sun, Feb 25, 2018 at 9:54 AM, Magnus Hagander wrote: >> Also if that wasn't clear -- we only do the full page write if there isn't >> already a checksum on the page and that checksum is correct. > > Hmm. > > Suppose that on the mast

Re: CALL optional in PL/pgSQL

2018-02-28 Thread David G. Johnston
On Wednesday, February 28, 2018, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > > > I seem to recall that there were past discussions about this, with > respect to the PERFORM command, but I couldn't find them anymore. > I'm thinking you are thinking of this one. https://www.postgr

Why chain of snapshots is used in ReorderBufferCommit?

2018-02-28 Thread Arseny Sher
Hello, While prowling through snapbuild & reorderbuffer code, I wondered: why a queue of snapshots is used for replaying each transaction instead of just picking up snapshot from snapbuilder once when COMMIT record is read? I am not aware of any DDL/DML mix which would make this invalid: e.g. we c

RE: [bug fix] Produce a crash dump before main() on Windows

2018-02-28 Thread Tsunakawa, Takayuki
From: Tsunakawa, Takayuki [mailto:tsunakawa.ta...@jp.fujitsu.com] > Another idea to add to the current patch is to move the call to SetErrorMode() > to the below function, which is called first in main(). How about this? > > void > pgwin32_install_crashdump_handler(void) > { > SetUnhandledE

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
On Thu, Mar 1, 2018 at 6:57 AM, Amit Langote wrote: > On 2018/02/28 19:14, Ashutosh Bapat wrote: >> On Wed, Feb 28, 2018 at 6:42 AM, Amit Langote wrote: >>> BTW, should there be a relevant test in partition_join.sql? If yes, >>> attached a patch (partitionwise-join-collation-test-1.patch) to add

CALL optional in PL/pgSQL

2018-02-28 Thread Peter Eisentraut
This seems to be a popular issue when porting from PL/SQL, so I'll throw it out here for discussion. Apparently, in PL/SQL you can call another procedure without the CALL keyword. Here is a patch that attempts to implement that in PL/pgSQL as well. It's not very pretty. I seem to recall that th

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-28 Thread Thomas Munro
On Mon, Feb 26, 2018 at 6:37 PM, Thomas Munro wrote: > I've now broken it into two patches. Rebased. -- Thomas Munro http://www.enterprisedb.com 0001-Enable-parallel-query-with-SERIALIZABLE-isolatio-v13.patch Description: Binary data 0002-Enable-the-read-only-SERIALIZABLE-optimization-f-v13

fixing more format truncation issues

2018-02-28 Thread Peter Eisentraut
In 6275f5d28a1577563f53f2171689d4f890a46881, we fixed warnings from the options -Wformat-overflow and -Wformat-truncation, which are part of -Wall in gcc 7. Here, I propose to dial this up a bit by adding -Wformat-overflow=2 -Wformat-truncation=2, which use some more worst-case approaches for esti

chained transactions

2018-02-28 Thread Peter Eisentraut
This feature is meant to help manage transaction isolation in procedures. I proposed elsewhere a patch that allows running SET TRANSACTION in PL/pgSQL. But if you have complex procedures that commit many times in different branches perhaps, you'd have to do this in every new transaction, which wo

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-02-28 Thread David Gould
On Wed, 28 Feb 2018 15:55:19 +0300 Alexander Kuzmenkov wrote: > Hi David, > > I was able to reproduce the problem using your script. > analyze_counts.awk is missing, though. Attached now I hope. I think I also added it to the commitfest page. > The idea of using the result of ANALYZE as-is,

Re: Support for ECDSA & ed25519 digital signatures in pgcrypto?

2018-02-28 Thread Bruce Momjian
On Sun, Feb 4, 2018 at 04:38:24PM +0530, Nilesh Trivedi wrote: > I recently had to build ed25519 digital signature validation in PostgreSQL. > Since pgcrypto doesn't > support these methods, I had to look into PL/Python and PL/v8 based > implementations. The > experience turned out to be very poor

faster testing with symlink installs

2018-02-28 Thread Peter Eisentraut
I'm proposing a way to make test runs a bit faster. A fair amount of time is taken by creating the test installation. Not huge compared to the whole test run, but still long enough to get boring. The idea (that I stole from somewhere else) is that we make the installation as symlinks pointing ba

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Amit Langote
On 2018/03/01 2:23, Robert Haas wrote: > On Tue, Feb 27, 2018 at 8:12 PM, Amit Langote > wrote: >> Ah, OK. I was missing that there is no need to have both parttypcoll and >> partcollation in PartitionSchemeData, as the Vars in rel->partexprs are >> built from a bare PartitionKey (not PartitionSch

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-02-28 Thread Michael Paquier
On Thu, Mar 01, 2018 at 01:26:32AM +, Tsunakawa, Takayuki wrote: > BTW, should pg_rewind really copy WAL files from the primary? If the > sole purpose of pg_rewind is to recover an instance to use as a > standby, can pg_rewind just remove all WAL files in the target > directory, because the st

Synchronous replay take III

2018-02-28 Thread Thomas Munro
Hi hackers, I was pinged off-list by a fellow -hackers denizen interested in the synchronous replay feature and wanting a rebased patch to test. Here it goes, just in time for a Commitfest. Please skip to the bottom of this message for testing notes. In previous threads[1][2][3] I called this f

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Amit Langote
On 2018/02/28 19:14, Ashutosh Bapat wrote: > On Wed, Feb 28, 2018 at 6:42 AM, Amit Langote wrote: >> BTW, should there be a relevant test in partition_join.sql? If yes, >> attached a patch (partitionwise-join-collation-test-1.patch) to add one. > > A partition-wise join path will be created but d

[bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-02-28 Thread Tsunakawa, Takayuki
Hello, Our customer hit another bug of pg_rewind with PG 9.5. The attached patch fixes this. PROBLEM After a long run of successful pg_rewind, the synchronized standby could not catch up the primary forever, emitting the following message repeatedly:

Re: Online enabling of checksums

2018-02-28 Thread Craig Ringer
On 1 March 2018 at 03:42, Alvaro Herrera wrote: > I noticed that pg_verify_checksum takes an "-o oid" argument to only > check the relation with that OID; but that's wrong, because the number > is a relfilenode, not an OID (since it's compared to the on-disk file > name). I would suggest changin

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Craig Ringer
On 1 March 2018 at 09:00, Justin Pryzby wrote: > > > > - started in single user mode or with system indices disabled? > > why? > > Some of these I suggested just as a datapoint (or other brainstorms I > couldn't > immediately reject). A cluster where someone has UPDATED pg_* (even > pg_statist

Re: Online enabling of checksums

2018-02-28 Thread Daniel Gustafsson
> On 01 Mar 2018, at 05:07, Tomas Vondra wrote: > > On 02/28/2018 08:42 PM, Alvaro Herrera wrote: >> I noticed that pg_verify_checksum takes an "-o oid" argument to only >> check the relation with that OID; but that's wrong, because the number >> is a relfilenode, not an OID (since it's compared

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Craig Ringer
On 1 March 2018 at 06:28, Justin Pryzby wrote: > On Wed, Feb 28, 2018 at 02:18:12PM -0800, Andres Freund wrote: > > On 2018-02-28 17:14:18 -0500, Peter Eisentraut wrote: > > > I can see why you'd want that, but as a DBA, I don't necessarily want > > > all of that recorded, especially in a quasi-p

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Craig Ringer
On 1 March 2018 at 05:43, Andres Freund wrote: > Hi, > > a significant number of times during investigations of bugs I wondered > whether running the cluster with various settings, or various tools > could've caused the issue at hand. Therefore I'd like to propose adding > a 'tainted' field to p

Re: Two small patches for the isolationtester lexer

2018-02-28 Thread Daniel Gustafsson
> On 01 Mar 2018, at 06:01, Tom Lane wrote: > Daniel Gustafsson writes: >> I agree, patch 0002 was broken and the correct fix is a much bigger project - >> one too big for me to tackle right now (but hopefully at some point in the >> near >> future). Thanks for the review of it though! > > OK

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Justin Pryzby
On Wed, Feb 28, 2018 at 02:23:19PM -0800, Andres Freund wrote: > Hi, > > On 2018-02-28 16:16:53 -0600, Justin Pryzby wrote: > > - did recovery (you could use "needed recovery" instead, but then there's > > the > >question of how reliable that field would be); > >+ or: timestamp of most

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-02-28 Thread Amit Langote
On 2018/03/01 1:03, Robert Haas wrote: > On Tue, Feb 27, 2018 at 7:46 PM, Alvaro Herrera > wrote: >> I updated Amit Langote's patch for INSERT ON CONFLICT DO UPDATE[1]. >> Following the lead of edd44738bc88 ("Be lazier about partition tuple >> routing.") this incarnation only does the necessary pu

Re: row filtering for logical replication

2018-02-28 Thread Craig Ringer
On 1 March 2018 at 07:03, Euler Taveira wrote: > Hi, > > The attached patches add support for filtering rows in the publisher. > The output plugin will do the work if a filter was defined in CREATE > PUBLICATION command. An optional WHERE clause can be added after the > table name in the CREATE P

Re: VPATH build from a tarball fails with some gmake versions

2018-02-28 Thread Andrew Dunstan
On Thu, Mar 1, 2018 at 9:24 AM, Tom Lane wrote: > I tried doing a VPATH build referencing a source directory that I'd > distclean'd > but not maintainer-clean'd, which should simulate the case of a VPATH > build from a tarball. I was quite surprised that it fell over: > > ... > gcc -Wall -Wmissi

Re: row filtering for logical replication

2018-02-28 Thread David Fetter
On Wed, Feb 28, 2018 at 08:03:02PM -0300, Euler Taveira wrote: > Hi, > > The attached patches add support for filtering rows in the publisher. > The output plugin will do the work if a filter was defined in CREATE > PUBLICATION command. An optional WHERE clause can be added after the > table name

Re: prokind column (was Re: [HACKERS] SQL procedures)

2018-02-28 Thread Michael Paquier
On Wed, Feb 28, 2018 at 05:37:11PM -0500, Peter Eisentraut wrote: > On 2/28/18 15:45, Tom Lane wrote: >> I have reviewed this patch and attach an updated version below. >> I've rebased it up to today, fixed a few minor errors, and adopted >> most of Michael's suggestions. Also, since I remain desp

Re: IndexTupleDSize macro seems redundant

2018-02-28 Thread Tom Lane
Stephen Frost writes: > Updated (combined) patch attached for review. I went through and looked > again to make sure there weren't any cases of making an unaligned > pointer to a struct and didn't see any, and I added some comments to > _bt_restore_page(). This seems to have fallen through a cra

Re: Cast jsonb to numeric, int, float, bool

2018-02-28 Thread Nikita Glukhov
On 01.03.2018 00:43, Darafei Praliaskouski wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested We're using

Re: Missing comment edit

2018-02-28 Thread Tom Lane
Kyotaro HORIGUCHI writes: > I happend to find that the comment on formdesc is missing > pg_subscription. Please find the attached patch (I'm sure:) to > fix that . Hmm ... certainly, that comment is now wrong, but I'm kind of inclined to just remove it, because it'll certainly be wrong again in f

unused includes in test_decoding

2018-02-28 Thread Euler Taveira
Hi, This is a cosmetic patch that removes some unused includes in test_decoding. It seems to be like this since day 1 (9.4). -- Euler Taveira Timbira - http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento From c5

Re: compress method for spgist - 2

2018-02-28 Thread Tom Lane
Alexander Korotkov writes: > On Thu, Jan 4, 2018 at 1:17 AM, Dagfinn Ilmari Mannsåker > wrote: >> This patch added two copies of the poly_ops row to the "Built-in SP-GiST >> Operator Classes" table in spgist.sgml. > Thank for fixing this! I'm sure that Teodor will push this after end of > New Y

row filtering for logical replication

2018-02-28 Thread Euler Taveira
Hi, The attached patches add support for filtering rows in the publisher. The output plugin will do the work if a filter was defined in CREATE PUBLICATION command. An optional WHERE clause can be added after the table name in the CREATE PUBLICATION such as: CREATE PUBLICATION foo FOR TABLE depart

Re: [HACKERS] pgbench randomness initialization

2018-02-28 Thread Tom Lane
Fabien COELHO writes: >> This is a simple patch that does what it says on the tin. I ran into >> trouble with the pgbench TAP test *even before applying the patch*, but >> only because I was doing a VPATH build as a user without 'write' >> on the source tree (001_pgbench_with_server.pl tried to ma

VPATH build from a tarball fails with some gmake versions

2018-02-28 Thread Tom Lane
I tried doing a VPATH build referencing a source directory that I'd distclean'd but not maintainer-clean'd, which should simulate the case of a VPATH build from a tarball. I was quite surprised that it fell over: ... gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -W

Re: prokind column (was Re: [HACKERS] SQL procedures)

2018-02-28 Thread Peter Eisentraut
On 2/28/18 15:45, Tom Lane wrote: > I have reviewed this patch and attach an updated version below. > I've rebased it up to today, fixed a few minor errors, and adopted > most of Michael's suggestions. Also, since I remain desperately > unhappy with putting zeroes into prorettype, I changed it to

INOUT parameters in procedures

2018-02-28 Thread Peter Eisentraut
This patch set adds support for INOUT parameters to procedures. Currently, INOUT and OUT parameters are not supported. A top-level CALL returns the output parameters as a result row. In PL/pgSQL, I have added special support to pass the output back into the variables, as one would expect. These

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Justin Pryzby
On Wed, Feb 28, 2018 at 02:18:12PM -0800, Andres Freund wrote: > On 2018-02-28 17:14:18 -0500, Peter Eisentraut wrote: > > I can see why you'd want that, but as a DBA, I don't necessarily want > > all of that recorded, especially in a quasi-permanent way. > > Huh? You're arguing that we should ma

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Andres Freund
Hi, On 2018-02-28 16:16:53 -0600, Justin Pryzby wrote: Unfortunately your list seems to raise the bar to a place I don't see us going soon :( > - pg_control versions used on this cluster (hopefully a full list..obviously >not going back before PG11); That needs arbitrary much space, that'

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-28 Thread Peter Geoghegan
On Fri, Feb 9, 2018 at 6:36 AM, Robert Haas wrote: > Here's my $0.02: I think that new concurrency errors thrown by the > merge code itself deserve strict scrutiny and can survive only if they > have a compelling justification, but if the merge code happens to > choose an action that leads to a co

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Andres Freund
Hi, On 2018-02-28 17:14:18 -0500, Peter Eisentraut wrote: > I can see why you'd want that, but as a DBA, I don't necessarily want > all of that recorded, especially in a quasi-permanent way. Huh? You're arguing that we should make it easier for DBAs to hide potential causes of corruption? I fail

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Justin Pryzby
On Wed, Feb 28, 2018 at 01:43:11PM -0800, Andres Freund wrote: > a significant number of times during investigations of bugs I wondered > whether running the cluster with various settings, or various tools > could've caused the issue at hand. Therefore I'd like to propose adding > a 'tainted' fiel

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Andres Freund
On 2018-02-28 23:13:44 +0100, Tomas Vondra wrote: > > On 02/28/2018 10:43 PM, Andres Freund wrote: > > Hi, > > > > a significant number of times during investigations of bugs I wondered > > whether running the cluster with various settings, or various tools > > could've caused the issue at hand.

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Peter Eisentraut
On 2/28/18 16:43, Andres Freund wrote: > a significant number of times during investigations of bugs I wondered > whether running the cluster with various settings, or various tools > could've caused the issue at hand. Therefore I'd like to propose adding > a 'tainted' field to pg_control, that co

Re: RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Tomas Vondra
On 02/28/2018 10:43 PM, Andres Freund wrote: > Hi, > > a significant number of times during investigations of bugs I wondered > whether running the cluster with various settings, or various tools > could've caused the issue at hand. Therefore I'd like to propose adding > a 'tainted' field to pg_

Re: [HACKERS] [PATCH] kNN for SP-GiST

2018-02-28 Thread Nikita Glukhov
Attached 3rd version of kNN for SP-GiST. On 09.03.2017 16:52, Alexander Korotkov wrote: Hi, Nikita! I take a look to this patchset.  My first notes are following. * 0003-Extract-index_store_orderby_distances-v02.patch Function index_store_orderby_distances doesn't look general enough for i

Re: Two small patches for the isolationtester lexer

2018-02-28 Thread Tom Lane
Daniel Gustafsson writes: > On 22 Feb 2018, at 05:10, Tom Lane wrote: >> Actually, looking closer, this would also trigger on '#' used inside a >> SQL literal, which seems to move the problem cases into the "pretty >> likely" category instead of the "far-fetched" one. So I'd only be OK >> with i

Re: Two small patches for the isolationtester lexer

2018-02-28 Thread Tom Lane
Daniel Gustafsson writes: >> On 22 Feb 2018, at 05:12, Tom Lane wrote: >> Another idea is just to teach addlitchar to realloc the buffer bigger >> when necessary. > I think this is the best approach for the task, the attached patch changes the > static allocation to instead realloc when required

Re: Cast jsonb to numeric, int, float, bool

2018-02-28 Thread Darafei Praliaskouski
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested We're using this patch and like it a lot. We store a lot of log-

RFC: Add 'taint' field to pg_control.

2018-02-28 Thread Andres Freund
Hi, a significant number of times during investigations of bugs I wondered whether running the cluster with various settings, or various tools could've caused the issue at hand. Therefore I'd like to propose adding a 'tainted' field to pg_control, that contains some of the "history" of the cluste

Re: Online enabling of checksums

2018-02-28 Thread Tomas Vondra
On 02/28/2018 08:42 PM, Alvaro Herrera wrote: > I noticed that pg_verify_checksum takes an "-o oid" argument to only > check the relation with that OID; but that's wrong, because the number > is a relfilenode, not an OID (since it's compared to the on-disk file > name). I would suggest changing ev

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-28 Thread Peter Geoghegan
On Wed, Feb 28, 2018 at 8:53 AM, Robert Haas wrote: > On Tue, Feb 27, 2018 at 5:07 PM, Peter Geoghegan wrote: >> I now feel like Simon's suggestion of throwing an error in corner >> cases isn't so bad. It still seems like we could do better, but the >> more I think about it, the less that seems l

SET TRANSACTION in PL/pgSQL

2018-02-28 Thread Peter Eisentraut
Currently, you can't run SET TRANSACTION in PL/pgSQL. A normal SQL command run inside PL/pgSQL acquires a snapshot, but SET TRANSACTION does not work anymore if a snapshot is set. Here is a patch to work around that by handling this command separately. I have coded this here bypassing SPI entire

2018-03 Commitfest starts tomorrow

2018-02-28 Thread David Steele
Hackers! I'll be starting the Commitfest at midnight AoE (07:00 ET, 13:00 CET) so please get your patches in before then. Please remember that if you drop a new and large (or invasive patch) into this CF it may be moved to the next CF. This last CF for PG11 should generally be restricted to patc

PATCH: Unlogged tables re-initialization tests

2018-02-28 Thread David Steele
These tests were originally included in the exclude unlogged tables patch [1] to provide coverage for the refactoring of reinit.c. After review we found a simpler implementation that did not require the reinit.c refactor so I dropped the tests from that patch. I did not include the refactor here

PL/pgSQL nested CALL with transactions

2018-02-28 Thread Peter Eisentraut
So far, a nested CALL or DO in PL/pgSQL would not establish a context where transaction control statements were allowed. This patch fixes that by handling CALL and DO specially in PL/pgSQL, passing the atomic/nonatomic execution context through and doing the required management around transaction

Re: Online enabling of checksums

2018-02-28 Thread Alvaro Herrera
I noticed that pg_verify_checksum takes an "-o oid" argument to only check the relation with that OID; but that's wrong, because the number is a relfilenode, not an OID (since it's compared to the on-disk file name). I would suggest changing everything to clarify that it's a pg_class.relfilenode v

Re: handling of heap rewrites in logical decoding

2018-02-28 Thread Peter Eisentraut
On 2/25/18 07:27, Craig Ringer wrote: > I'm pretty sure we _will_ want the ability to decode and stream rewrite > contents for non-IMMUTABLE table rewrites. > > Filtering out by default is OK by me, but I think making it impossible > to decode is a mistake. So I'm all for the oid option and had wr

Re: [PATCH] Verify Checksums during Basebackups

2018-02-28 Thread David Steele
On 2/28/18 1:08 PM, Michael Banck wrote: > > The attached small patch verifies checksums (in case they are enabled) > during a basebackup. The rationale is that we are reading every block in > this case anyway, so this is a good opportunity to check them as well. > Other and complementary ways of

Re: [HACKERS] Cast jsonb to numeric, int, float, bool

2018-02-28 Thread Anastasia Lubennikova
01.02.2017 17:41, Anastasia Lubennikova: Now the simplest way to extract booleans and numbers from json/jsonb is to cast it to text and then cast to the appropriate type: postgres=# select 'true'::jsonb::text::bool;  bool --  t postgres=# select '1.0'::jsonb::text::numeric;  numeric ---

Re: Server won't start with fallback setting by initdb.

2018-02-28 Thread Robert Haas
On Fri, Feb 9, 2018 at 3:08 AM, Kyotaro HORIGUCHI wrote: > I'm not sure such a case happens in the real world nowadays, > initdb uses the fallback value of max_connections=10. But it is > out of favor of server since it is not larger than > max_wal_senders(10). > >> postgres: max_wal_senders must

[PATCH] Verify Checksums during Basebackups

2018-02-28 Thread Michael Banck
Hi, some installations have data which is only rarerly read, and if they are so large that dumps are not routinely taken, data corruption would only be detected with some large delay even with checksums enabled. The attached small patch verifies checksums (in case they are enabled) during a baseb

Re: Parallel index creation & pg_stat_activity

2018-02-28 Thread Andres Freund
Hi Peter, On 2018-02-28 16:50:44 +, Phil Florent wrote: > With an index creation (create index t1_i1 on t1(c1, c2);) I have this kind > of output : > > ./t -d 20 -o "pid, backend_type, query, wait_event_type, wait_event" > busy_pc | distinct_exe | pid | backend_type | query

Re: Changing the autovacuum launcher scheduling; oldest table first algorithm

2018-02-28 Thread Grigory Smolkin
On 02/28/2018 12:04 PM, Masahiko Sawada wrote: Hi, I've created the new thread for the changing AV launcher scheduling. The problem of AV launcher scheduling is described on [1] but I summarize it here. If there is even one database that is at risk of wraparound, currently AV launcher selects

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 8:12 PM, Amit Langote wrote: > Ah, OK. I was missing that there is no need to have both parttypcoll and > partcollation in PartitionSchemeData, as the Vars in rel->partexprs are > built from a bare PartitionKey (not PartitionSchemeData), and after that > point, parttypcoll

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-02-28 Thread Shubham Barai
On 28 February 2018 at 05:51, Thomas Munro wrote: > On Wed, Jan 3, 2018 at 4:31 AM, Shubham Barai > wrote: > > I have created new isolation tests. Please have a look at > > updated patch. > > Hi Shubham, > > Could we please have a rebased version of the gin one? > Sure. I have attached a rebase

Re: Online enabling of checksums

2018-02-28 Thread Robert Haas
On Sun, Feb 25, 2018 at 9:54 AM, Magnus Hagander wrote: > Also if that wasn't clear -- we only do the full page write if there isn't > already a checksum on the page and that checksum is correct. Hmm. Suppose that on the master there is a checksum on the page and that checksum is correct, but on

Re: [HACKERS] MERGE SQL Statement for PG11

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 5:07 PM, Peter Geoghegan wrote: > I now feel like Simon's suggestion of throwing an error in corner > cases isn't so bad. It still seems like we could do better, but the > more I think about it, the less that seems like a cop-out. My reasons > are: I still think we really

Re: [HACKERS] user-defined numeric data types triggering ERROR: unsupported type

2018-02-28 Thread Tomas Vondra
OK, time to revive this old thread ... On 09/23/2017 05:27 PM, Tom Lane wrote: > Tomas Vondra writes: [ scalarineqsel may fall over when used by extension operators ] > >> What about using two-pronged approach: > >> 1) fall back to mid bucket in back branches (9.3 - 10) >> 2) do something

Re: Registering LWTRANCHE_PARALLEL_HASH_JOIN

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 3:58 PM, Thomas Munro wrote: > On Wed, Feb 28, 2018 at 8:39 AM, Robert Haas wrote: >> On Sat, Feb 10, 2018 at 6:07 PM, Thomas Munro >> wrote: >>> I forgot to register a display name for LWTRANCHE_PARALLEL_HASH_JOIN, >>> the tranche ID used by the LWLock that Parallel Hash

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 7:46 PM, Alvaro Herrera wrote: > I updated Amit Langote's patch for INSERT ON CONFLICT DO UPDATE[1]. > Following the lead of edd44738bc88 ("Be lazier about partition tuple > routing.") this incarnation only does the necessary push-ups for the > specific partition that needs

Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-02-28 Thread Anastasia Lubennikova
Hi, I want to propose a bunch of patches which allow to reduce WAL traffic generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree and RUM, we can now log index pages of other access methods only once in the end of indexbuild process. Implementation is based on generic_xlog. Not

Re: server crash in nodeAppend.c

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 5:24 AM, Rajkumar Raghuwanshi wrote: > SET parallel_setup_cost=0; > SET parallel_tuple_cost=0; > create or replace function foobar() returns setof text as > $$ select 'foo'::varchar union all select 'bar'::varchar ; $$ > language sql stable; > > postgres=# select foobar();

PATCH: Exclude temp relations from base backup

2018-02-28 Thread David Steele
This is a follow-up patch from the exclude unlogged relations discussion [1]. The patch excludes temporary relations during a base backup using the existing looks_like_temp_rel_name() function for identification. It shares code to identify database directories from [1], so for now that has been d

Re: Let's remove DSM_INPL_NONE.

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 11:30 PM, Tom Lane wrote: > I'd be in favor of having some normally-ifdef'd-out facility for causing > failures of this kind. (As I mentioned upthread, I do not think "always > fail" is sufficient.) That's very different from having a user-visible > option, though. We do

Re: Kerberos test suite

2018-02-28 Thread Peter Eisentraut
On 2/27/18 00:56, Thomas Munro wrote: > FWIW it passes for me if I add this: > > +elsif ($^O eq 'freebsd') > +{ > + $krb5_bin_dir = '/usr/local/bin'; > + $krb5_sbin_dir = '/usr/local/sbin'; I suppose you only need the second one, right? The first one should be in the path. > +} > >

RE: Direct converting numeric types to bool

2018-02-28 Thread Alex Ignatov
-Original Message- From: n.zhuch...@postgrespro.ru [mailto:n.zhuch...@postgrespro.ru] Sent: Wednesday, February 28, 2018 6:04 PM To: pgsql-hackers Subject: Direct converting numeric types to bool Attached patch allow direct convertion of numeric types to bool like integer::bool. Support

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 5:14 PM, Tom Lane wrote: >> I ran the postgres_fdw regression test with no sleep two times in a >> CLOBBER_CACHE_ALWAYS-enabled build, and then the regression test with >> the sleep (60 seconds) two times, but I couldn't reproduce that in both >> cases. I suspect the chang

Re: Direct converting numeric types to bool

2018-02-28 Thread Pavel Stehule
2018-02-28 16:13 GMT+01:00 Pavel Stehule : > Hi > > 2018-02-28 16:06 GMT+01:00 : > >> n.zhuch...@postgrespro.ru писал 2018-02-28 18:04: >> >> Attached patch allow direct convertion of numeric types to bool like >>> integer::bool. >>> Supported types: >>> - smallint; >>> - bigint; >>> - real; >>

Re: Direct converting numeric types to bool

2018-02-28 Thread Pavel Stehule
Hi 2018-02-28 16:06 GMT+01:00 : > n.zhuch...@postgrespro.ru писал 2018-02-28 18:04: > > Attached patch allow direct convertion of numeric types to bool like >> integer::bool. >> Supported types: >> - smallint; >> - bigint; >> - real; >> - double precision; >> - decimal(numeric). >> >> This f

Re: Incorrect comments in partition.c

2018-02-28 Thread Robert Haas
On Wed, Feb 28, 2018 at 3:24 AM, Etsuro Fujita wrote: > I'll add this to the upcoming commitfest. Committed. Sorry that I didn't notice this thread sooner (and that the original commits didn't take care of it). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: Direct converting numeric types to bool

2018-02-28 Thread Tom Lane
n.zhuch...@postgrespro.ru writes: > Attached patch allow direct convertion of numeric types to bool like > integer::bool. > Supported types: > - smallint; > - bigint; > - real; > - double precision; > - decimal(numeric). > This functionality is helped with migration from Oracle. I think

Re: [HACKERS] [POC] Faster processing at Gather node

2018-02-28 Thread Robert Haas
On Tue, Feb 27, 2018 at 4:06 PM, Andres Freund wrote: >> OK, I'll try to check how feasible that would be. > > cool. It's not too hard, but it doesn't really seem to help, so I'm inclined to leave it alone. To make it work, you need to keep two separate counters in the shm_mq_handle, one for the

Re: Direct converting numeric types to bool

2018-02-28 Thread n . zhuchkov
n.zhuch...@postgrespro.ru писал 2018-02-28 18:04: Attached patch allow direct convertion of numeric types to bool like integer::bool. Supported types: - smallint; - bigint; - real; - double precision; - decimal(numeric). This functionality is helped with migration from Oracle. diff --git a/

Direct converting numeric types to bool

2018-02-28 Thread n . zhuchkov
Attached patch allow direct convertion of numeric types to bool like integer::bool. Supported types: - smallint; - bigint; - real; - double precision; - decimal(numeric). This functionality is helped with migration from Oracle. -- Nikita Zhuchkov Postgres Professional: http://www.postgresp

Re: Disabling src/test/[ssl|ldap] when not building with SSL/LDAP support

2018-02-28 Thread Peter Eisentraut
On 2/24/18 18:29, Michael Paquier wrote: > Sure. But then I think that it would be nice to show up on screen the > reason why the test failed if possible. As of now if SSL is missing the > whole run shows in red without providing much useful information. > Instead of 0001 as shaped previously, wh

  1   2   >