Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 06:43:31PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > On Mon, Jul 8, 2019 at 05:41:51PM -0400, Stephen Fro

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-07-08 Thread Thomas Munro
On Tue, Jul 2, 2019 at 5:46 PM Paul Guo wrote: > Rebased, aligned with recent changes in pg_rewind/pg_basebackup and then > retested. Thanks. Hi Paul, A minor build problem on Windows: src/bin/pg_rewind/pg_rewind.c(32): fatal error C1083: Cannot open include file: 'backup_common.h': No such fi

Re: Avoiding deadlock errors in CREATE INDEX CONCURRENTLY

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 10:33 AM Goel, Dhruv wrote: > I have attached the revised patch. I ran check-world multiple times on my > machine and it seems to succeed now. Do you mind kicking-off the CI build > with the latest patch? Thanks. It's triggered automatically when you post patches to the

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jul 8, 2019 at 06:43:31PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Mon, Jul 8, 2019 at 06:04:46PM -0400, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > > On Mo

Re: Extra quote_all_identifiers in _dumpOptions

2019-07-08 Thread Bruce Momjian
On Thu, Jun 27, 2019 at 12:12:15PM +0300, Arthur Zakirov wrote: > Hello hackers, > > While working on pg_dump I noticed the extra quote_all_identifiers in > _dumpOptions struct. I attached the patch. > > It came from refactoring by 0eea8047bf. There is also a discussion: > https://www.postgresql.

Re: OpenSSL specific value under USE_SSL instead of USE_OPENSSL

2019-07-08 Thread Bruce Momjian
On Thu, Jun 27, 2019 at 04:02:45PM +0200, Daniel Gustafsson wrote: > The ssl_ciphers GUC for configuring cipher suites sets the default value based > on USE_SSL but it’s actually an OpenSSL specific value. As with other such > changes, it works fine now but will become an issue should we support o

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 07:27:12PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Operationally, how would that work? We unlock them all on boot but > > somehow make them inaccessible to some backends after that? > > That could work and doesn't seem like an insurmount

Re: Increasing default value for effective_io_concurrency?

2019-07-08 Thread Bruce Momjian
On Wed, Jul 3, 2019 at 11:42:49AM -0400, Robert Haas wrote: > On Wed, Jul 3, 2019 at 11:24 AM Tomas Vondra > wrote: > > Maybe. And it would probably work for the systems I used for benchmarks. > > > > It however assumes two things: (a) the storage system actually has > > spindles and (b) you know

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-08 Thread Bruce Momjian
On Mon, Jul 1, 2019 at 09:51:12AM -0400, Alvaro Herrera wrote: > Now that we have REINDEX CONCURRENTLY, I think reindexdb is going to > gain more popularity. > > Please don't reuse a file name as generic as "parallel.c" -- it's > annoying when navigating source. Maybe conn_parallel.c multiconn.c

Re: Improve search for missing parent downlinks in amcheck

2019-07-08 Thread Peter Geoghegan
On Sun, Jul 7, 2019 at 7:53 PM Thomas Munro wrote: > On Wed, May 1, 2019 at 12:58 PM Peter Geoghegan wrote: > > I will think about a simple fix, but after the upcoming point release. > > There is no hurry. > > A bureaucratic question: What should the status be for this CF entry? I have plans to

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Bruce Momjian
On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > > While I was fooling with it I noticed that the existing code for -n > > is buggy. The documentation says clearly that only the first > > argument is a candidate to be -n: > > > > If the first argument is an unquoted -n the

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 03:21:41PM -0400, Tom Lane wrote: > Having said that, join_hash.sql in particular seems to have zero > value if it's not testing hash joins, so I think it'd be reasonable > for it to override a global enable_hashjoin = off setting. None of > the other regression test script

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread Tomas Vondra
On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote: On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra wrote: On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote: >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra > wrote: >> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-08 Thread James Coleman
Note: As I was writing this, I saw a new email come in from Tomas with a new patch series, and some similar observations. I'll look at that patch series more, but I think it's likely far more complete, so will end up going with that. I wanted to send this email anyway to at least capture the debugg

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:19 AM Tom Lane wrote: > Given the purposes of this test, I think it'd be reasonable to force > both enable_hashjoin = on and enable_mergejoin = off at the very top > of the join_hash script, or the corresponding place in join.sql in > v11. Thomas, was there a specific rea

RE: [PATCH] Speedup truncates of relation forks

2019-07-08 Thread Jamison, Kirk
Hi Thomas, Thanks for checking. > On Fri, Jul 5, 2019 at 3:03 PM Jamison, Kirk wrote: > > I updated the patch which is similar to V3 of the patch, but > > addressing my problem in (5) in the previous email regarding > FreeSpaceMapVacuumRange. > > It seems to pass the regression test now. Kindly

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > On Mon, Jul 08, 2019 at 03:21:41PM -0400, Tom Lane wrote: >> Having said that, join_hash.sql in particular seems to have zero >> value if it's not testing hash joins, so I think it'd be reasonable >> for it to override a global enable_hashjoin = off setting. None of >> t

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Thomas Munro writes: > On Tue, Jul 9, 2019 at 2:19 AM Tom Lane wrote: >> Given the purposes of this test, I think it'd be reasonable to force >> both enable_hashjoin = on and enable_mergejoin = off at the very top >> of the join_hash script, or the corresponding place in join.sql in >> v11. Thom

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:22 PM Tom Lane wrote: > Thomas Munro writes: > > Based on a suggestion from Andres (if I recall correctly), I wrapped > > each individual test in savepoint/rollback, and then set just the GUCs > > needed to get the plan shape and execution code path I wanted to > > exerci

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Bruce Momjian
On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/11/ddl-partitioning.html > Description: > > In the documentation for Postgres 11 table partitioning, there is no me

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: > Yeah. I had obviously never noticed that test script. +1 for just > enabling hash joins the top of join_hash.sql in 12+, and the > equivalent section in 11's join.sql (which is luckily at the end of > the file). Right, I did not pay

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Tom Lane
Michael Paquier writes: > On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: >> Yeah. I had obviously never noticed that test script. +1 for just >> enabling hash joins the top of join_hash.sql in 12+, and the >> equivalent section in 11's join.sql (which is luckily at the end of >> t

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 10:37:37PM -0400, Bruce Momjian wrote: > On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: >> In the documentation for Postgres 11 table partitioning, there is no mention >> of the requirement that the Primary Key of a partitioned table must contain >> th

Re: doc: improve PG 12 to_timestamp()/to_date() wording

2019-07-08 Thread Bruce Momjian
On Sat, Jul 6, 2019 at 03:24:25PM -0500, Justin Pryzby wrote: > On Tue, Apr 30, 2019 at 07:14:04PM -0500, Justin Pryzby wrote: > > On Tue, Apr 30, 2019 at 09:48:14PM +0300, Alexander Korotkov wrote: > > > I'd like to add couple of comments from my side. > > > > > > - returns an error becaus

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Thomas Munro
On Tue, Jul 9, 2019 at 2:45 PM Michael Paquier wrote: > On Tue, Jul 09, 2019 at 02:30:51PM +1200, Thomas Munro wrote: > > Yeah. I had obviously never noticed that test script. +1 for just > > enabling hash joins the top of join_hash.sql in 12+, and the > > equivalent section in 11's join.sql (wh

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Tom Lane
Michael Paquier writes: > Attached is an idea of patch for the documentation, using this > wording: > + > + > + When defining a primary key on a partitioned table, the primary > + key column must be included in the partition key. > + > + Isn't it the other way ar

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread David G. Johnston
On Mon, Jul 8, 2019 at 7:59 PM Michael Paquier wrote: > Attached is an idea of patch for the documentation, using this > wording: > + > + > + When defining a primary key on a partitioned table, the primary > + key column must be included in the partition key. > + > +

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 03:04:27PM +1200, Thomas Munro wrote: > It's my mistake. I'll fix it later today. Thanks! -- Michael signature.asc Description: PGP signature

Re: pgbench - add \aset to store results of a combined query

2019-07-08 Thread Ibrar Ahmed
> SELECT 1 AS one \; > SELECT 2 AS two UNION SELECT 2 \; > SELECT 3 AS three \aset > > will set both "one" and "three", while "two" is not set because there were > two rows. It is a kind of more permissive \gset. Are you sure two is not set :)? SELECT 2 AS two UNION SELECT 2; -- only return

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Tom Lane
Bruce Momjian writes: > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: >>> I fixed that, but I'm wondering if we should back-patch that fix >>> or leave the back branches alone. >> +0.5 for back-patching. > Uh, if this was done in a major release I am thinking we have to mention >

Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-07-08 Thread Mike Palmiotto
On Mon, Jul 8, 2019 at 10:59 AM Mike Palmiotto wrote: > > On Sun, Jul 7, 2019 at 9:46 PM Thomas Munro wrote: >> >> On Sat, Apr 6, 2019 at 3:06 PM Andres Freund wrote: >> > I've moved this to the next CF, and marked it as targeting v13. >> >> Hi Mike, >> >> Commifest 1 for PostgreSQL 13 is here.

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread Bruce Momjian
On Mon, Jul 8, 2019 at 11:29:00PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > >>> I fixed that, but I'm wondering if we should back-patch that fix > >>> or leave the back branches alone. > > >> +0.5 for back-patching. > > >

Re: warning to publication created and wal_level is not set to logical

2019-07-08 Thread Lucas Viecelli
Hi Thomas. Attached is the rebased > The July Commitfest has started. This patch is in "Needs review" > status, but it doesn't apply. If I read the above discussion > correctly, it seems there is agreement that a warning here is a good > idea to commit this patch. Could you please post a reba

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Ryan Lambert
Hey everyone, Here is my input regarding nonces and randomness. > As I understand it, the NIST recommendation is a 96-bit *random* nonce, I could not find that exact requirement in the NIST documents, though given the volume of that library it would be possible to miss. The recommendation I rep

Re: [PATCH v4] Add \warn to psql

2019-07-08 Thread David G. Johnston
On Mon, Jul 8, 2019 at 8:35 PM Bruce Momjian wrote: > On Mon, Jul 8, 2019 at 11:29:00PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Fri, Jul 5, 2019 at 11:29:03PM +0200, David Fetter wrote: > > >>> I fixed that, but I'm wondering if we should back-patch that fix > > >>> or leave

Re: tableam vs. TOAST

2019-07-08 Thread Prabhat Sahu
On Mon, Jul 8, 2019 at 9:06 PM Robert Haas wrote: > On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu > wrote: > > I have tested the TOAST patches(v3) with different storage options > like(MAIN, EXTERNAL, EXTENDED, etc.), and > > combinations of compression and out-of-line storage options. > > I have

Re: range_agg

2019-07-08 Thread Pavel Stehule
Hi po 8. 7. 2019 v 18:47 odesílatel Paul A Jungwirth < p...@illuminatedcomputing.com> napsal: > On Sat, Jul 6, 2019 at 12:13 PM Jeff Davis wrote: > > > > On Fri, 2019-07-05 at 09:58 -0700, Paul A Jungwirth wrote: > > > user-defined range types. So how about I start on it and see how it > > > goe

Re: warning to publication created and wal_level is not set to logical

2019-07-08 Thread Lucas Viecelli
Follow the correct file, I added the wrong patch in the previous email > Attached is the rebased > thanks a lot *Lucas Viecelli* diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c index 1ac1a71bd9..902180cedc 100644 --- a/src/backend/commands/publicati

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Rajkumar Raghuwanshi
On Tue, Jul 9, 2019 at 8:29 AM Michael Paquier wrote: > On Mon, Jul 08, 2019 at 10:37:37PM -0400, Bruce Momjian wrote: > > On Fri, Jul 5, 2019 at 09:20:07PM +, PG Doc comments form wrote: > >> In the documentation for Postgres 11 table partitioning, there is no > mention > >> of the requirem

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 08:12:18PM -0700, David G. Johnston wrote: > Reads a bit backward. How about: > > "As uniqueness can only be enforced within an individual partition when > defining a primary key on a partitioned table all columns present in the > partition key must also exist in the prima

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-08 Thread Masahiko Sawada
On Tue, Jul 9, 2019 at 3:39 AM Tomas Vondra wrote: > > BTW how do you know this is what users want? Maybe they do, but then > again - maybe they just see it as magic and don't realize the extra > complexity (not just at the database level). In my experience users > generally want more abstract thi

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Michael Paquier
On Mon, Jul 08, 2019 at 11:10:51PM -0400, Tom Lane wrote: > Isn't it the other way around, that the partition key column(s) must > be > included in the primary key? Maybe I'm confused, but it seems like > we couldn't enforce PK uniqueness otherwise. Yes you are right. The full column list of the

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-08 Thread Michael Paquier
On Tue, Jul 09, 2019 at 03:34:48PM +0900, Michael Paquier wrote: > Looking closely at the code in DefineIndex() (and as Rajkumar has > mentioned upthread for unique constraints) this can happen for primary > keys, unique constraints and exclusion constraints. So we had better > mention all three o

<    1   2