Re: Problem with default partition pruning

2019-08-06 Thread Amit Langote
On Wed, Aug 7, 2019 at 3:30 PM yuzuko wrote: > > In short, I propose to get this done as the patch I posted in > > https://postgr.es/m/20190806133053.GA23706@alvherre.pgsql > > > I agree with your proposal. Also, I confirmed a default partition was pruned > as expected with your patch. +1. Than

Re: Problem with default partition pruning

2019-08-06 Thread yuzuko
Hello, > > Well, if this is really all that duplicative, one thing we could do is > > run this check in get_partprune_steps_internal only if > > constraint_exclusion is a value other than on; we should achieve the > > same effect with no repetition. Patch for that is attached. However, > > if I

Re: [PATCH] Absolute passwordfile path

2019-08-06 Thread Danylo Hlynskyi
Yes, I've resent it to pgpool-hack...@pgpool.net Sorry for the noise ср, 7 серп. 2019 о 09:18 Amit Langote пише: > Hello, > > On Wed, Aug 7, 2019 at 3:05 PM Danylo Hlynskyi > wrote: > > > > The pool_passwd option [1] is specified relative to config file. But for > greater flexibility absolute

Re: [PATCH] Absolute passwordfile path

2019-08-06 Thread Amit Langote
Hello, On Wed, Aug 7, 2019 at 3:05 PM Danylo Hlynskyi wrote: > > The pool_passwd option [1] is specified relative to config file. But for > greater flexibility absolute path should be accepted as well. > > If pool_passwd option starts with /, let's treat it as absolute path. > Otherwise, it is

Re: proposal: type info support functions for functions that use "any" type

2019-08-06 Thread Pavel Stehule
Hi pá 26. 7. 2019 v 22:53 odesílatel Tom Lane napsal: > I wrote: > > TBH, I don't like this proposal one bit. As far as I can see, the idea > > is to let a function's support function redefine the function's declared > > argument and result types on-the-fly according to no predetermined rules,

Fwd: [PATCH] Absolute passwordfile path

2019-08-06 Thread Danylo Hlynskyi
The pool_passwd option [1] is specified relative to config file. But for greater flexibility absolute path should be accepted as well. If pool_passwd option starts with /, let's treat it as absolute path. Otherwise, it is treated as relative path. Patch attached. Original author - Derek Kulinski

Re: POC: Cleaning up orphaned files using undo logs

2019-08-06 Thread Thomas Munro
Hi, I'll be responding to a bunch of long review emails in this thread point by point separately, but just picking out a couple of points here that jumped out at me: On Wed, Aug 7, 2019 at 9:18 AM Andres Freund wrote: > > + { > > + /* > > +

Re: proposal: type info support functions for functions that use "any" type

2019-08-06 Thread Pavel Stehule
čt 1. 8. 2019 v 11:01 odesílatel Thomas Munro napsal: > On Sat, Jul 27, 2019 at 5:45 PM Pavel Stehule > wrote: > > pá 26. 7. 2019 v 22:53 odesílatel Tom Lane napsal: > >> I wrote: > >> > TBH, I don't like this proposal one bit. As far as I can see, the > idea > >> > is to let a function's supp

Re: no default hash partition

2019-08-06 Thread Amit Langote
Horiguchi-san, On Wed, Aug 7, 2019 at 1:59 PM Kyotaro Horiguchi wrote: > At Tue, 6 Aug 2019 23:26:19 -0400, Robert Haas wrote: > > On Tue, Aug 6, 2019 at 6:58 PM Tom Lane wrote: > > I think, as Amit says, that having an automatic partition creation > > feature for hash partitions (and maybe oth

is necessary to recheck cached data in fn_extra?

2019-08-06 Thread Pavel Stehule
Hi I should to use a cache accessed via fn_extra. There will be stored data about function parameters (types). If I understand correctly, these data should be stable in query, and then recheck is not necessary. Is it true? Regards Pavel

Re: stress test for parallel workers

2019-08-06 Thread Thomas Munro
On Wed, Aug 7, 2019 at 5:07 PM Tom Lane wrote: > Thomas Munro writes: > > Another question is whether the build farm should be setting the Linux > > oom score adjust thing. > > AFAIK you can't do that without being root. Rats, yeah you need CAP_SYS_RESOURCE or root to lower it. -- Thomas Munro

Re: stress test for parallel workers

2019-08-06 Thread Tom Lane
Thomas Munro writes: > Another question is whether the build farm should be setting the Linux > oom score adjust thing. AFAIK you can't do that without being root. regards, tom lane

Re: stress test for parallel workers

2019-08-06 Thread Thomas Munro
On Wed, Aug 7, 2019 at 4:29 PM Tom Lane wrote: > Thomas Munro writes: > > I wondered if the build farm should try to report OOM kill -9 or other > > signal activity affecting the postmaster. > > Yeah, I've been wondering whether pg_ctl could fork off a subprocess > that would fork the postmaster,

Re: no default hash partition

2019-08-06 Thread Kyotaro Horiguchi
At Tue, 6 Aug 2019 23:26:19 -0400, Robert Haas wrote in > On Tue, Aug 6, 2019 at 6:58 PM Tom Lane wrote: > I think, as Amit says, that having an automatic partition creation > feature for hash partitions (and maybe other kinds, but certainly for > hash) would be a useful thing to add to the sys

Re: stress test for parallel workers

2019-08-06 Thread Tom Lane
Thomas Munro writes: > I wondered if the build farm should try to report OOM kill -9 or other > signal activity affecting the postmaster. Yeah, I've been wondering whether pg_ctl could fork off a subprocess that would fork the postmaster, wait for the postmaster to exit, and then report the exit

Re: Built-in connection pooler

2019-08-06 Thread Li Japin
Hi, Konstantin I test the patch-16 on postgresql master branch, and I find the temporary table cannot removed when we re-connect to it. Here is my test: japin@ww-it:~/WwIT/postgresql/Debug/connpool$ initdb The files belonging to this database system will be owned by user "japin". This user must

Re: no default hash partition

2019-08-06 Thread David Fetter
On Tue, Aug 06, 2019 at 06:58:44PM -0400, Tom Lane wrote: > Alvaro Herrera writes: > > On 2019-Aug-06, Tom Lane wrote: > >> Seems like "it's likely to cause trouble for users" is just going to > >> beg the question "why?". Can we explain the hazard succinctly? > >> Or point to a comment somewhere

Re: no default hash partition

2019-08-06 Thread Robert Haas
On Tue, Aug 6, 2019 at 6:58 PM Tom Lane wrote: > Hmm. So given the point about it being hard to predict which hash > partitions would receive what values ... under what circumstances > would it be sensible to not create a full set of partitions? Should > we just enforce that there is a full set,

Re: Built-in connection pooler

2019-08-06 Thread Ryan Lambert
Hi Konstantin, I did some testing with the latest patch [1] on a small local VM with 1 CPU and 2GB RAM with the intention of exploring pg_pooler_state(). Configuration: idle_pool_worker_timeout = 0 (default) connection_proxies = 2 max_sessions = 10 (default) max_connections = 1000 Initialized p

Re: FETCH FIRST clause PERCENT option

2019-08-06 Thread Kyotaro Horiguchi
Hello. At Thu, 1 Aug 2019 15:54:11 +0300, Surafel Temesgen wrote in > > Other than that, we can rip the clause if it is 100% > > > > > > You mean if PERCENT=100 it should short circuit and run the query > > normally? I like that. > > > > The patch did not did it automatically. Its query write

Re: partition routing layering in nodeModifyTable.c

2019-08-06 Thread Etsuro Fujita
Hi, On Wed, Aug 7, 2019 at 11:47 AM Amit Langote wrote: > On Wed, Aug 7, 2019 at 11:30 AM Etsuro Fujita wrote: > > On Wed, Aug 7, 2019 at 10:24 AM Amit Langote > > wrote: > > > * Regarding setting ForeignScan.resultRelIndex even for non-direct > > > modifications, maybe that's not a good idea

Re: remove "msg" parameter from convert_tuples_by_name

2019-08-06 Thread Amit Langote
On Wed, Aug 7, 2019 at 7:47 AM Alvaro Herrera wrote: > > Hello, here's a pretty trivial cleanup. > > Currently, you have to pass the errmsg text to convert_tuples_by_name > and convert_tuples_by_position that's going to be raised if the tuple > descriptors don't match. In the latter's case that m

Re: partition routing layering in nodeModifyTable.c

2019-08-06 Thread Amit Langote
Fujita-san, Thanks for the quick follow up. On Wed, Aug 7, 2019 at 11:30 AM Etsuro Fujita wrote: > On Wed, Aug 7, 2019 at 10:24 AM Amit Langote wrote: > > * Regarding setting ForeignScan.resultRelIndex even for non-direct > > modifications, maybe that's not a good idea anymore. A foreign table

Re: no default hash partition

2019-08-06 Thread Amit Langote
Hi, On Wed, Aug 7, 2019 at 8:02 AM Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Alvaro Herrera writes: > > > On 2019-Aug-06, Tom Lane wrote: > > >> Seems like "it's likely to cause trouble for users" is just going to > > >> beg the question "why?". Can we explain the hazard

Re: partition routing layering in nodeModifyTable.c

2019-08-06 Thread Etsuro Fujita
Amit-san, On Wed, Aug 7, 2019 at 10:24 AM Amit Langote wrote: > On Tue, Aug 6, 2019 at 9:56 PM Etsuro Fujita wrote: > > What > > I'm thinking for the setrefs.c change is to modify ForeignScan (ie, > > set_foreignscan_references) rather than ModifyTable, like the > > attached. > > Thanks for the

Re: no default hash partition

2019-08-06 Thread Amit Langote
Hi Alvaro, On Wed, Aug 7, 2019 at 7:27 AM Alvaro Herrera wrote: > > Given the discussion starting at > https://postgr.es/m/cafjfprdbiqjzm8sg9+s0x8re-afhds6mflgguw0wvunlgrv...@mail.gmail.com > we don't have default-partition support with the hash partitioning > scheme. That seems a reasonable out

Re: More issues with pg_verify_checksums and checksum verification in base backups

2019-08-06 Thread Michael Paquier
On Tue, Aug 06, 2019 at 04:20:43PM -0400, Stephen Frost wrote: > On Tue, Aug 6, 2019 at 15:45 Magnus Hagander wrote: >> When agreement cannot be found, perhaps a parameter is in order? >> >> That is, have the tool complain about such files by default but with a >> HINT that it may or may not be a

Re: Assertion for logically decoding multi inserts into the catalog

2019-08-06 Thread Michael Paquier
On Tue, Aug 06, 2019 at 03:08:48PM +0200, Daniel Gustafsson wrote: > Thanks, this is a much better approach and it passes tests for me. +1 on this > version (regardless of outcome of the other patch as this is separate). I had an extra lookup at this stuff this morning, and applied the patch. Pl

Re: partition routing layering in nodeModifyTable.c

2019-08-06 Thread Amit Langote
Fujita-san, Thanks a lot the review. On Tue, Aug 6, 2019 at 9:56 PM Etsuro Fujita wrote: > On Mon, Aug 5, 2019 at 6:16 PM Amit Langote wrote: > > I first thought to set it > > only if direct modification is being used, but maybe it'd be simpler > > to set it even if direct modification is not u

Re: Refactoring code stripping trailing \n and \r from strings

2019-08-06 Thread Michael Paquier
On Tue, Aug 06, 2019 at 03:10:33PM -0400, Bruce Momjian wrote: > On Thu, Aug 1, 2019 at 12:18:20PM +0900, Michael Paquier wrote: >> b654714 has reworked the way we handle removal of CLRF for several >> code paths, and has repeated the same code patterns to do that in 8 >> different places. Could

Re: stress test for parallel workers

2019-08-06 Thread Thomas Munro
On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: > Thomas Munro writes: > > On Wed, Jul 24, 2019 at 10:11 AM Tom Lane wrote: > > Do you have an example to hand? Is this > > failure always happening on Linux? > > I dug around a bit further, and while my recollection of a lot of > "postmaster exit

Re: no default hash partition

2019-08-06 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> Hmm. So given the point about it being hard to predict which hash > >> partitions would receive what values ... under what circumstances > >> would it be sensible to not crea

Re: no default hash partition

2019-08-06 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Hmm. So given the point about it being hard to predict which hash >> partitions would receive what values ... under what circumstances >> would it be sensible to not create a full set of partitions? Should >> we just enforce that

Re: partition routing layering in nodeModifyTable.c

2019-08-06 Thread Andres Freund
Hi, On 2019-08-05 18:16:10 +0900, Amit Langote wrote: > The patch adds a resultRelIndex field to ForeignScan node, which is > set to >= 0 value for non-SELECT queries. I first thought to set it > only if direct modification is being used, but maybe it'd be simpler > to set it even if direct modif

Re: dropdb --force

2019-08-06 Thread Ryan Lambert
I set the status to Waiting on Author since Tom's concerns [1] have not been addressed. [1] https://www.postgresql.org/message-id/15707.1564024305%40sss.pgh.pa.us Thanks, Ryan

Re: no default hash partition

2019-08-06 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Alvaro Herrera writes: > > On 2019-Aug-06, Tom Lane wrote: > >> Seems like "it's likely to cause trouble for users" is just going to > >> beg the question "why?". Can we explain the hazard succinctly? > >> Or point to a comment somewhere else t

Re: no default hash partition

2019-08-06 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-06, Tom Lane wrote: >> Seems like "it's likely to cause trouble for users" is just going to >> beg the question "why?". Can we explain the hazard succinctly? >> Or point to a comment somewhere else that explains it? > Right ... the "trouble" is just that if t

Re: no default hash partition

2019-08-06 Thread Alvaro Herrera
On 2019-Aug-06, Tom Lane wrote: > Alvaro Herrera writes: > > Given the discussion starting at > > https://postgr.es/m/cafjfprdbiqjzm8sg9+s0x8re-afhds6mflgguw0wvunlgrv...@mail.gmail.com > > we don't have default-partition support with the hash partitioning > > scheme. That seems a reasonable outc

Re: FETCH FIRST clause PERCENT option

2019-08-06 Thread Ryan Lambert
Surafel, The patch did not did it automatically. Its query writer obligation to do > that currently Ok. Your latest patch [1] passes make installcheck-world, I didn't test the actual functionality this round. plan = (Plan *) make_limit(plan, > subparse->limitOffset, > - subparse->limi

remove "msg" parameter from convert_tuples_by_name

2019-08-06 Thread Alvaro Herrera
Hello, here's a pretty trivial cleanup. Currently, you have to pass the errmsg text to convert_tuples_by_name and convert_tuples_by_position that's going to be raised if the tuple descriptors don't match. In the latter's case that makes sense, as each case is pretty specific and tailored messages

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Bruce Momjian
On Tue, Aug 6, 2019 at 06:13:30PM -0400, Jonathan Katz wrote: > Hi, > > On 8/6/19 3:01 PM, Bruce Momjian wrote: > > On Tue, Aug 6, 2019 at 01:55:38PM -0400, Bruce Momjian wrote: > >> CTR mode creates a bit stream for the first 16 bytes with nonce of > >> (segment_number, counter = 0), and the ne

Re: no default hash partition

2019-08-06 Thread Tom Lane
Alvaro Herrera writes: > Given the discussion starting at > https://postgr.es/m/cafjfprdbiqjzm8sg9+s0x8re-afhds6mflgguw0wvunlgrv...@mail.gmail.com > we don't have default-partition support with the hash partitioning > scheme. That seems a reasonable outcome, but I think we should have a > comment

no default hash partition

2019-08-06 Thread Alvaro Herrera
Given the discussion starting at https://postgr.es/m/cafjfprdbiqjzm8sg9+s0x8re-afhds6mflgguw0wvunlgrv...@mail.gmail.com we don't have default-partition support with the hash partitioning scheme. That seems a reasonable outcome, but I think we should have a comment about it (I had to search the rea

Re: Problem with default partition pruning

2019-08-06 Thread Alvaro Herrera
On 2019-Aug-06, Alvaro Herrera wrote: > Well, if this is really all that duplicative, one thing we could do is > run this check in get_partprune_steps_internal only if > constraint_exclusion is a value other than on; we should achieve the > same effect with no repetition. Patch for that is attach

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Jonathan S. Katz
Hi, On 8/6/19 3:01 PM, Bruce Momjian wrote: > On Tue, Aug 6, 2019 at 01:55:38PM -0400, Bruce Momjian wrote: >> CTR mode creates a bit stream for the first 16 bytes with nonce of >> (segment_number, counter = 0), and the next 16 bytes with >> (segment_number, counter = 1), etc. We only XOR using

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-06 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Ian Barwick writes: > >>> I dislike the special-casing of ALTER SYSTEM here, where we're basically > >>> saying that only ALTER SYSTEM is allowed to do this cleanup and that if > >>> such cleanup is wanted then ALTER SYSTEM must be run. > > > T

Re: POC: Cleaning up orphaned files using undo logs

2019-08-06 Thread Andres Freund
Hi, On 2019-08-06 00:56:26 -0700, Andres Freund wrote: > Out of energy. Here's the last section of my low-leve review. Plan to write a higher level summary afterwards, now that I have a better picture of the code. > +static void > +UndoDiscardOneLog(UndoLogSlot *slot, TransactionId xmin, bool *

Re: Cleanup of intro.sgml

2019-08-06 Thread Joshua D. Drake
Rev 2 attached. Added: SQL/JSON SQL/XML Fixed spelling mistakes Fixed a missing closing tag. -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc Postgres centered full stack support, consulting and development. Advocate: @amplifypostgres || Get help: https://commandp

Re: More issues with pg_verify_checksums and checksum verification in base backups

2019-08-06 Thread Stephen Frost
Greetings, On Tue, Aug 6, 2019 at 15:45 Magnus Hagander wrote: > On Tue, Aug 6, 2019 at 6:07 PM Stephen Frost wrote: > >> Greetings, >> >> * Andres Freund (and...@anarazel.de) wrote: >> > On 2019-08-06 10:58:15 -0400, Stephen Frost wrote: >> > > * Michael Banck (michael.ba...@credativ.de) wrote

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-06 Thread Tom Lane
Ian Barwick writes: > On 8/6/19 11:16 AM, Stephen Frost wrote: >>> Erm, those are duplicates though and we're saying that ALTER SYSTEM >>> removes those... Seems like we should be normalizing the file to be >>> consistent in this regard too. > True. (Switches brain on)... Ah yes, with the patch

Small patch to fix build on Windows

2019-08-06 Thread Dmitry Igrishin
Hi, The attached self-documented patch fixes build on Windows in case when path to Python has embedded spaces. diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm index d1d0aed07e..76834f5188 100644 --- a/src/tools/msvc/Mkvcbuild.pm +++ b/src/tools/msvc/Mkvcbuild.pm @@ -495,7 +4

Re: More issues with pg_verify_checksums and checksum verification in base backups

2019-08-06 Thread Magnus Hagander
On Tue, Aug 6, 2019 at 6:07 PM Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2019-08-06 10:58:15 -0400, Stephen Frost wrote: > > > * Michael Banck (michael.ba...@credativ.de) wrote: > > > > Independently of the whitelist/blacklist question, I believe > >

Re: Refactoring code stripping trailing \n and \r from strings

2019-08-06 Thread Bruce Momjian
On Thu, Aug 1, 2019 at 12:18:20PM +0900, Michael Paquier wrote: > Hi Tom, > > b654714 has reworked the way we handle removal of CLRF for several > code paths, and has repeated the same code patterns to do that in 8 > different places. Could it make sense to refactor things as per the > attached

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Bruce Momjian
On Tue, Aug 6, 2019 at 01:55:38PM -0400, Bruce Momjian wrote: > CTR mode creates a bit stream for the first 16 bytes with nonce of > (segment_number, counter = 0), and the next 16 bytes with > (segment_number, counter = 1), etc. We only XOR using the parts of the > bit stream we want to use. We

Re: intarray GiST index gets wrong answers for '{}' <@ anything

2019-08-06 Thread Tom Lane
Alexander Korotkov writes: > Users, who likes existing behavior of handling <@ operator in intarray > opclasses, may be advised to rewrite their queries as following. > "col <@ const" => "col <@ const AND col && const" Oh, that's a good suggestion --- it will work, and work reasonably well, with

Re: block-level incremental backup

2019-08-06 Thread Ibrar Ahmed
On Tue, Aug 6, 2019 at 11:31 PM Ibrar Ahmed wrote: > > I have not looked at the patch in detail, but just some nits from my side. > > On Fri, Aug 2, 2019 at 6:13 PM vignesh C wrote: > >> On Thu, Aug 1, 2019 at 5:06 PM Jeevan Chalke >> wrote: >> > >> > On Tue, Jul 30, 2019 at 9:39 AM Jeevan Chal

Re: block-level incremental backup

2019-08-06 Thread Ibrar Ahmed
I have not looked at the patch in detail, but just some nits from my side. On Fri, Aug 2, 2019 at 6:13 PM vignesh C wrote: > On Thu, Aug 1, 2019 at 5:06 PM Jeevan Chalke > wrote: > > > > On Tue, Jul 30, 2019 at 9:39 AM Jeevan Chalke < > jeevan.cha...@enterprisedb.com> wrote: > >> > >> I am almo

Re: How am I supposed to fix this?

2019-08-06 Thread Larry Rosenman
On 08/06/2019 1:16 pm, Peter Geoghegan wrote: On Tue, Aug 6, 2019 at 11:11 AM Larry Rosenman wrote: As a followup, btcheck found another index that had issues, and a toast table was missing a chunk. I have ALL the data I used to create this table still around so I just dropped it and am reloa

Re: intarray GiST index gets wrong answers for '{}' <@ anything

2019-08-06 Thread Alexander Korotkov
Hi! On Tue, Aug 6, 2019 at 8:56 PM Tom Lane wrote: > The reason appears to be that the condition for descending through a > non-leaf index key for the RTContainedBy case is incorrectly optimistic: > it supposes that we only need to descend into subtrees whose union key > overlaps the query array.

Re: How am I supposed to fix this?

2019-08-06 Thread Peter Geoghegan
On Tue, Aug 6, 2019 at 11:11 AM Larry Rosenman wrote: > As a followup, btcheck found another index that had issues, and a toast > table was missing a chunk. > > I have ALL the data I used to create this table still around so I just > dropped it and am reloading the data. It sounds like there is a

Re: How am I supposed to fix this?

2019-08-06 Thread Larry Rosenman
On 08/06/2019 12:45 pm, Larry Rosenman wrote: On 08/06/2019 12:35 pm, Peter Geoghegan wrote: On Tue, Aug 6, 2019 at 10:34 AM Larry Rosenman wrote: ERROR: function bt_index_check(index => oid) does not exist LINE 1: SELECT bt_index_check(index => c.oid), ^ HINT: No function ma

Re: SQL:2011 PERIODS vs Postgres Ranges?

2019-08-06 Thread Ibrar Ahmed
On Tue, Aug 6, 2019 at 8:28 PM Paul Jungwirth wrote: > Hi Ibrar, > > On 8/6/19 3:26 AM, Ibrar Ahmed wrote: > > - Why we are not allowing any other datatype other than ranges in the > > primary key. Without that there is no purpose of a primary key. > > A temporal primary key always has at least o

Re: [PATCH] Atomic pgrename on Windows

2019-08-06 Thread Alexander Korotkov
Hi! I'd like to resume the discussion on this subject. Sorry for so long delay. On Sat, Jan 20, 2018 at 6:13 PM Magnus Hagander wrote: > On Tue, Nov 28, 2017 at 2:47 AM, Michael Paquier > wrote: >> >> On Mon, Nov 27, 2017 at 3:28 PM, Alexander Korotkov >> wrote: >> > Attached patch atomic-pg

intarray GiST index gets wrong answers for '{}' <@ anything

2019-08-06 Thread Tom Lane
While looking at the pending patch for faster GIN index searches on no-key queries, I was motivated to improve contrib/intarray's regression test to exercise the GIN_SEARCH_MODE_ALL case, because it didn't. And then I thought well, let's try to bring the code coverage of _int_gin.c up to something

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Bruce Momjian
On Wed, Aug 7, 2019 at 12:31:58AM +0900, Masahiko Sawada wrote: > Well, so you mean that for example we encrypt only 100 bytes WAL > record when append 100 bytes WAL records? > > For WAL encryption, if we encrypt the entire 8k WAL page and write the > entire page, the encrypted-and-written page w

Re: Cleanup of intro.sgml

2019-08-06 Thread Joshua D. Drake
On 8/5/19 1:13 PM, Chapman Flack wrote: On 8/5/19 3:20 PM, Joshua D. Drake wrote: intro.sgml today. Patch attached. Things I noticed quickly: broken up in to categoriess/in to/into/ Got it, I can make that change. Unstructured data via JSON(or XML ?) On this one, there is a lot

Re: How am I supposed to fix this?

2019-08-06 Thread Larry Rosenman
On 08/06/2019 12:35 pm, Peter Geoghegan wrote: On Tue, Aug 6, 2019 at 10:34 AM Larry Rosenman wrote: ERROR: function bt_index_check(index => oid) does not exist LINE 1: SELECT bt_index_check(index => c.oid), ^ HINT: No function matches the given name and argument types. You m

Re: How am I supposed to fix this?

2019-08-06 Thread Peter Geoghegan
On Tue, Aug 6, 2019 at 10:34 AM Larry Rosenman wrote: > ERROR: function bt_index_check(index => oid) does not exist > LINE 1: SELECT bt_index_check(index => c.oid), > ^ > HINT: No function matches the given name and argument types. You might > need to add explicit type casts. It

Re: How am I supposed to fix this?

2019-08-06 Thread Larry Rosenman
On 08/06/2019 12:30 pm, Peter Geoghegan wrote: On Tue, Aug 6, 2019 at 10:19 AM Tomas Vondra wrote: The question is how much other data corruption is there ... Larry could try running amcheck on the other indexes. Just the basic bt_check_index() checks should be enough to detect problems like

Re: How am I supposed to fix this?

2019-08-06 Thread Peter Geoghegan
On Tue, Aug 6, 2019 at 10:19 AM Tomas Vondra wrote: > The question is how much other data corruption is there ... Larry could try running amcheck on the other indexes. Just the basic bt_check_index() checks should be enough to detect problems like this. They can be run fairly non-disruptively. So

Re: How am I supposed to fix this?

2019-08-06 Thread Alvaro Herrera
On 2019-Aug-06, Larry Rosenman wrote: > ler=# reindex index pg_toast_17760_index; > ERROR: relation "pg_toast_17760_index" does not exist Maybe try "reindex index pg_toast.pg_toast_17760_index" > ler=# reindex (verbose) database ler; [...] > ERROR: index "pg_toast_17760_index" contains unexpec

Re: How am I supposed to fix this?

2019-08-06 Thread Tomas Vondra
On Tue, Aug 06, 2019 at 12:06:45PM -0500, Larry Rosenman wrote: I'm getting the below, and am unaware of how to fix it 11.4 on FreeBSD 12. ler=# reindex (verbose) table dns_query ; INFO: index "dns_query_pkey" was reindexed DETAIL: CPU: user: 114.29 s, system: 207.94 s, elapsed: 698.87

Re: Problem with default partition pruning

2019-08-06 Thread Alvaro Herrera
Hello, On 2019-Aug-06, Amit Langote wrote: > On Mon, Aug 5, 2019 at 11:39 PM Alvaro Herrera > wrote: > > I don't think that we care about what happens with constraint_exclusion > > is on. That's not the recommended value for that setting anyway. > > One issue I expressed with unconditionally

How am I supposed to fix this?

2019-08-06 Thread Larry Rosenman
I'm getting the below, and am unaware of how to fix it 11.4 on FreeBSD 12. ler=# reindex (verbose) table dns_query ; INFO: index "dns_query_pkey" was reindexed DETAIL: CPU: user: 114.29 s, system: 207.94 s, elapsed: 698.87 s ERROR: index "pg_toast_17760_index" contains unexpected zero p

Re: The unused_oids script should have a reminder to use the 8000-8999 OID range

2019-08-06 Thread Alvaro Herrera
On 2019-Aug-06, Tom Lane wrote: > My estimate is that in any one development > cycle we'll commit order-of-a-couple-dozen patches that consume new OIDs. > In that context you'd be just unlucky to get an OID suggestion that > doesn't have dozens to hundreds of free OIDs after it. (If the rate > of

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Masahiko Sawada
Hi, On Wed, Aug 7, 2019, 00:31 Masahiko Sawada wrote: > Hi Bruce, > (off-list) > > I think I'm missing something about basic of encryption. Please let me > question about it on off-list. > Sorry for the noise, it was not off-list. I made a mistake. > On Tue, Aug 6, 2019 at 11:36 PM Bruce Momj

Re: More issues with pg_verify_checksums and checksum verification in base backups

2019-08-06 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-08-06 10:58:15 -0400, Stephen Frost wrote: > > * Michael Banck (michael.ba...@credativ.de) wrote: > > > Independently of the whitelist/blacklist question, I believe > > > pg_checksums should not error out as soon as it encounters a w

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Masahiko Sawada
Hi Bruce, (off-list) I think I'm missing something about basic of encryption. Please let me question about it on off-list. On Tue, Aug 6, 2019 at 11:36 PM Bruce Momjian wrote: > > On Tue, Aug 6, 2019 at 12:00:27PM +0900, Masahiko Sawada wrote: > > What I'm thinking about WAL encryption is that

Re: Use PageIndexTupleOverwrite() within nbtsort.c

2019-08-06 Thread Anastasia Lubennikova
16.07.2019 1:12, Peter Geoghegan wrote: Attached patch slightly simplifies nbtsort.c by making it use PageIndexTupleOverwrite() to overwrite the last right non-pivot tuple with the new high key (pivot tuple). PageIndexTupleOverwrite() is designed so that code like this doesn't need to delete and

Re: SQL:2011 PERIODS vs Postgres Ranges?

2019-08-06 Thread Paul Jungwirth
Hi Ibrar, On 8/6/19 3:26 AM, Ibrar Ahmed wrote: - Why we are not allowing any other datatype other than ranges in the primary key. Without that there is no purpose of a primary key. A temporal primary key always has at least one ordinary column (of any type), so it is just a traditional prima

Re: More issues with pg_verify_checksums and checksum verification in base backups

2019-08-06 Thread Andres Freund
Hi, On 2019-08-06 10:58:15 -0400, Stephen Frost wrote: > * Michael Banck (michael.ba...@credativ.de) wrote: > > Independently of the whitelist/blacklist question, I believe > > pg_checksums should not error out as soon as it encounters a weird looking > > file, but either (i) still checksum it or

Re: More issues with pg_verify_checksums and checksum verification in base backups

2019-08-06 Thread Stephen Frost
Greetings, * Michael Banck (michael.ba...@credativ.de) wrote: > Independently of the whitelist/blacklist question, I believe > pg_checksums should not error out as soon as it encounters a weird looking > file, but either (i) still checksum it or (ii) skip it? Or is that to be > considered a pilot

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-06 Thread Bruce Momjian
On Tue, Aug 6, 2019 at 12:00:27PM +0900, Masahiko Sawada wrote: > What I'm thinking about WAL encryption is that WAL records on WAL > buffer is not encrypted. When writing to the disk we copy the contents > of 8k WAL page to a temporary buffer and encrypt it, and then write > it. And according to

Re: Problem with default partition pruning

2019-08-06 Thread Alvaro Herrera
On 2019-Aug-05, Alvaro Herrera wrote: > So we have three locations for that test; one is where it currently is, > which handles a small subset of the cases. The other is where Amit > first proposed putting it, which handles some additional cases; and the > third one is where your latest patch put

Re: Assertion for logically decoding multi inserts into the catalog

2019-08-06 Thread Daniel Gustafsson
> On 6 Aug 2019, at 05:36, Michael Paquier wrote: > > On Tue, Aug 06, 2019 at 12:52:09AM +0200, Daniel Gustafsson wrote: >> Yeah, this is clearly fat-fingered, the intent is to only run the Assert in >> case XLH_INSERT_CONTAINS_NEW_TUPLE is set in xlrec->flags, as it only applies >> under that co

Re: partition routing layering in nodeModifyTable.c

2019-08-06 Thread Etsuro Fujita
Amit-san, On Mon, Aug 5, 2019 at 6:16 PM Amit Langote wrote: > On Sat, Aug 3, 2019 at 3:01 AM Andres Freund wrote: > Based on the discussion, I have updated the patch. > > > I'm a bit woried about the move of BeginDirectModify() into > > nodeModifyTable.c - it just seems like an odd control flow

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-06 Thread Thomas Munro
On Tue, Aug 6, 2019 at 9:26 PM Heikki Linnakangas wrote: > I had some steam, and wrote a spec that reproduces this bug. It wasn't > actually that hard to reproduce, fortunately. Or unfortunately; people > might well be hitting it in production. I used the "freezetest.spec" > from the 2013 thread a

Re: SQL:2011 PERIODS vs Postgres Ranges?

2019-08-06 Thread Ibrar Ahmed
Hi Paul, On Mon, Aug 5, 2019 at 3:11 AM Paul A Jungwirth wrote: > On Fri, Aug 2, 2019 at 1:49 PM Ibrar Ahmed wrote: > > I did some clean-up on this patch. I have also refactored a small > portion of the code > > to reduce the footprint of the patch. For simplicity, I have divided the > patch in

Re: Identity columns should own only one sequence

2019-08-06 Thread Peter Eisentraut
On 2019-08-05 13:30, Laurenz Albe wrote: > Peter Eisentraut wrote: >> On 2019-05-08 16:49, Laurenz Albe wrote: >>> I believe we should have both: >>> >>> - Identity columns should only use sequences with an INTERNAL dependency, >>> as in Peter's patch. >> >> I have committed this. > > Since this

Update to DocBook 4.5

2019-08-06 Thread Peter Eisentraut
I propose to apply the attached patch (to master) to update the DocBook version to 4.5 (from 4.2). This basically just gets us off some random intermediate minor version to the latest within that major version. Most packagings put all 4.* versions into one package, so you probably don't need to c

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-06 Thread Heikki Linnakangas
On 06/08/2019 07:20, Thomas Munro wrote: On Tue, Aug 6, 2019 at 4:35 AM Andres Freund wrote: On 2019-08-05 20:58:05 +1200, Thomas Munro wrote: 1. Commit dafaa3efb75 "Implement genuine serializable isolation level." (2011) locked the root tuple, because it set t_self to *tid. Possibly due to c

Re: Fix a typo in add_partial_path

2019-08-06 Thread Amit Langote
On Tue, Aug 6, 2019 at 6:12 PM Michael Paquier wrote: > > On Tue, Aug 06, 2019 at 05:34:06PM +0900, Amit Langote wrote: > > Attached patch for: > > > > s/incompable/incompatible/g > > Thanks, applied. Thank you Michael. Regards, Amit

Re: Fix a typo in add_partial_path

2019-08-06 Thread Michael Paquier
On Tue, Aug 06, 2019 at 05:34:06PM +0900, Amit Langote wrote: > Attached patch for: > > s/incompable/incompatible/g Thanks, applied. -- Michael signature.asc Description: PGP signature

Re: Index Skip Scan

2019-08-06 Thread Floris Van Nee
> Yes, the check should be for that. However, the query in question > doesn't have any query_pathkeys, and hence query_uniquekeys in > standard_qp_callback(), so therefore it isn't supported > Your scenario is covered by one of the test cases in case the > functionality is supported. But, I think

Fix a typo in add_partial_path

2019-08-06 Thread Amit Langote
Hi, Attached patch for: s/incompable/incompatible/g Thanks, Amit add_partial_path-typo.patch Description: Binary data

Re: Global temporary tables

2019-08-06 Thread Konstantin Knizhnik
New version of the patch with several fixes is attached. Many thanks to Roman Zharkov for testing. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_bufferca

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-06 Thread Kuntal Ghosh
Hello Thomas, On Tue, Aug 6, 2019 at 9:50 AM Thomas Munro wrote: > > On Tue, Aug 6, 2019 at 4:35 AM Andres Freund wrote: > > On 2019-08-05 20:58:05 +1200, Thomas Munro wrote: > > > 1. Commit dafaa3efb75 "Implement genuine serializable isolation > > > level." (2011) locked the root tuple, becaus

Re: BUG #15938: Corrupted WAL segment after crash recovery

2019-08-06 Thread Kyotaro Horiguchi
Hello. At Thu, 01 Aug 2019 13:52:52 +, PG Bug reporting form wrote in <15938-8591df7e95064...@postgresql.org> > The following bug has been logged on the website: > > Bug reference: 15938 > Logged by: Alexander Kukushkin > Email address: cyberd...@gmail.com > PostgreSQL ve

Re: POC: Cleaning up orphaned files using undo logs

2019-08-06 Thread Andres Freund
Hi, On 2019-08-05 11:29:34 -0700, Andres Freund wrote: > Need to do something else for a bit. More later. Here we go. > + /* > + * Compute the header size of the undo record. > + */ > +Size > +UndoRecordHeaderSize(uint16 uur_info) > +{ > + Sizesize; > + > + /* Add fixed

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-06 Thread Konstantin Knizhnik
On 05.08.2019 22:35, Daniel Migowski wrote: . I think that including in pg_prepared_statements information about memory used this statement is very useful. CachedPlanMemoryUsage function may be useful not only for this view, but for example it is also need in my autoprepare patch. I would lo

  1   2   >