Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-09 Thread Stephen Frost
Greetings, * Ryan Lambert (r...@rustprooflabs.com) wrote: > > What I think Tomas is getting at here is that we don't write a page only > > once. > > > A nonce of tableoid+pagenum will only be unique the first time we write > > out that page. Seems unlikely that we're only going to be writing the

Re: Implementing Incremental View Maintenance

2019-07-09 Thread Takuma Hoshiai
Hi Thomas, 2019年7月8日(月) 15:32 Thomas Munro : > On Fri, Jun 28, 2019 at 10:56 PM Yugo Nagata wrote: > > Attached is a WIP patch of IVM which supports some aggregate functions. > > Hi Nagata-san and Hoshiai-san, > > Thank you for working on this. I enjoyed your talk at PGCon. I've > added Kevin

Re: coypu: "FATAL: sorry, too many clients already"

2019-07-09 Thread Tom Lane
Thomas Munro writes: > Several times on master[1] beginning with an initial occurrence 36 > days ago, and every time on REL_12_STABLE[2], but not on older > branches, build farm animal coypu has failed in the regression tests > with the error given in the subject. How can there be too many if > t

Re: Index Skip Scan

2019-07-09 Thread Thomas Munro
On Wed, Jul 10, 2019 at 1:32 AM Jesper Pedersen wrote: > > While updating the Loose Index Scan wiki page with links to other > > products' terminology on this subject, I noticed that MySQL can > > skip-scan MIN() and MAX() in the same query. Hmm. That seems quite > > desirable. I think it requi

Re: Implementing Incremental View Maintenance

2019-07-09 Thread Takuma Hoshiai
On Wed, 10 Jul 2019 11:07:15 +0900 Takuma Hoshiai wrote: > Hi Thomas, > > 2019年7月8日(月) 15:32 Thomas Munro : > > > On Fri, Jun 28, 2019 at 10:56 PM Yugo Nagata wrote: > > > Attached is a WIP patch of IVM which supports some aggregate functions. > > > > Hi Nagata-san and Hoshiai-san, > > > > Tha

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-09 Thread Ryan Lambert
If a random number were generated instead its result would need to be stored somewhere too, correct? > That might also allow things like backup software to work > on these encrypted data files for page-level backups without needing > access to the key and that'd be pretty neat. +1 Ryan

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-09 Thread Stephen Frost
Greetings, * Ryan Lambert (r...@rustprooflabs.com) wrote: > If a random number were generated instead its result would need to be > stored somewhere too, correct? Yes. Thanks, Stephen signature.asc Description: PGP signature

Re: Minimal logical decoding on standbys

2019-07-09 Thread Andres Freund
Hi, Thanks for the new version! Looks like we're making progress towards something committable here. I think it'd be good to split the patch into a few pieces. I'd maybe do that like: 1) WAL format changes (plus required other changes) 2) Recovery conflicts with slots 3) logical decoding on stand

Re: Minimal logical decoding on standbys

2019-07-09 Thread Robert Haas
On Tue, Jul 9, 2019 at 11:14 PM Andres Freund wrote: > > + if (xlrec->wal_level < WAL_LEVEL_LOGICAL) > > + ereport(ERROR, > > + > > (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), > > +

Re: FETCH FIRST clause PERCENT option

2019-07-09 Thread Ryan Lambert
I did some more testing. I initialized a database with 1 million rows with indexes and joins to test against and ran pgbench with a few different settings for % to return. I started with a base query not utilizing the new functionality. The queries used are similar to my prior examples, code at [

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-07-09 Thread Dilip Kumar
On Fri, Nov 2, 2018 at 1:34 PM Amit Langote wrote: > > On 2018/11/01 20:34, Dilip Kumar wrote: > > On Mon, Oct 29, 2018 at 2:53 PM Amit Langote wrote: > >> Anyway, why don't we just use the child table's AppendRelInfo to get the > >> parent's version of varattno instead of creating a new function?

Re: Remove page-read callback from XLogReaderState.

2019-07-09 Thread Kyotaro Horiguchi
Hello. The patch gets disliked by my tool chain. Fixed the usage of PG_USED_FOR_ASSERTS_ONLY and rebased to bd56cd75d2. -- Kyotaro Horiguchi NTT Open Source Software Center >From 1dabdce6993b73408b950cb8c348c4999178b9a0 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 18 Apr 2019 10:2

Re: range_agg

2019-07-09 Thread Paul A Jungwirth
On Tue, Jul 9, 2019 at 12:02 PM Jeff Davis wrote: > > - Multirange in/out work just like arrays, e.g. '{"[1,3)", "[5,6)"}' > > It would be cool to have a better text representation. We could go > simple like: > >'[1,3) [5,6)' Will that work with all ranges, even user-defined ones? With a tstz

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-07-09 Thread Kyotaro Horiguchi
Hello. Rebased the patch to master(bd56cd75d2). regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From ac52e2c1c56a96c1745149ff4220a3a116d6c811 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 11 Oct 2018 10:03:21 +0900 Subject: [PATCH 1/3] TAP test for copy-truncation o

Re: range_agg

2019-07-09 Thread Paul A Jungwirth
On Tue, Jul 9, 2019 at 12:24 PM Pavel Stehule wrote: > út 9. 7. 2019 v 21:10 odesílatel Pavel Stehule > napsal: >> I afraid so with generic multiragetype there lot of array infrastructure >> will be duplicated > > on second hand - it is true so classic array concat is not optimal for set of >

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-07-09 Thread Dilip Kumar
On Wed, Jul 10, 2019 at 9:44 AM Dilip Kumar wrote: > > On Fri, Nov 2, 2018 at 1:34 PM Amit Langote > wrote: > > > > On 2018/11/01 20:34, Dilip Kumar wrote: > > > On Mon, Oct 29, 2018 at 2:53 PM Amit Langote wrote: > > >> Anyway, why don't we just use the child table's AppendRelInfo to get the > >

Re: shared-memory based stats collector

2019-07-09 Thread Kyotaro Horiguchi
At Thu, 04 Jul 2019 19:27:54 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190704.192754.27063464.horikyota@gmail.com> > #db #tbl #clients #iter #xactlen #referers > A1: 1 1 1 2100 > A2: 1 1 1 2101 > B

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-09 Thread Michael Paquier
On Tue, Jul 09, 2019 at 11:54:29AM -0700, Melanie Plageman wrote: > It might be worth post-processing results files to ignore row ordering > in some cases to allow for easier comparison. Has this been proposed > in the past? Not that I recall. -- Michael signature.asc Description: PGP signature

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-07-09 Thread Amit Langote
Hi Dilip, On Wed, Jul 10, 2019 at 1:29 PM Dilip Kumar wrote: > On Wed, Jul 10, 2019 at 9:44 AM Dilip Kumar wrote: > > On Fri, Nov 2, 2018 at 1:34 PM Amit Langote wrote: > > > Okay, here are two patches: > > > > > > 0001 adds a new RelOptInfo member inh_root_parent that's set for > > > inheritanc

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-09 Thread Michael Paquier
On Tue, Jul 09, 2019 at 01:09:38PM +0200, Peter Eisentraut wrote: > You can already do that: Run a query through psql to get a list of > affected tables or indexes and feed those to reindexdb using -i or -t > options. Sure, but that's limited if one can only afford a limited amount of downtime for

Re: range_agg

2019-07-09 Thread Pavel Stehule
st 10. 7. 2019 v 6:26 odesílatel Paul A Jungwirth < p...@illuminatedcomputing.com> napsal: > On Tue, Jul 9, 2019 at 12:24 PM Pavel Stehule > wrote: > > út 9. 7. 2019 v 21:10 odesílatel Pavel Stehule > napsal: > >> I afraid so with generic multiragetype there lot of array > infrastructure will be

Re: SQL/JSON path issues/questions

2019-07-09 Thread Alexander Korotkov
On Mon, Jul 8, 2019 at 12:30 AM Alexander Korotkov wrote: > On Thu, Jul 4, 2019 at 4:38 PM Liudmila Mantrova > wrote: > > Thank you! > > > > I think we can make this sentence even shorter, the fix is attached: > > > > "To refer to a JSON element stored at a lower nesting level, add one or > > mo

Re: Postgres 11: Table Partitioning and Primary Keys

2019-07-09 Thread Amit Langote
On Wed, Jul 10, 2019 at 7:53 AM Alvaro Herrera wrote: > On 2019-Jul-09, Amit Langote wrote: > > "While defining a primary key and unique constraints on partitioned > > tables is supported, the set of columns being constrained must include > > all of the partition key columns. This limitation exis

Re: Optimize partial TOAST decompression

2019-07-09 Thread Binguo Bao
Tomas Vondra 于2019年7月10日周三 上午5:12写道: > On Sat, Jul 06, 2019 at 05:23:37PM +0200, Tomas Vondra wrote: > >On Sat, Jul 06, 2019 at 02:27:56AM +0800, Binguo Bao wrote: > >>Hi, Tomas! > >>Thanks for your testing and the suggestion. > >> > >>That's quite bizarre behavior - it does work with a prefix, b

doc: minor update for description of "pg_roles" view

2019-07-09 Thread Ian Barwick
Hi Here: https://www.postgresql.org/docs/12/view-pg-roles.html we state: "This view explicitly exposes the OID column of the underlying table, since that is needed to do joins to other catalogs." I think it's superfluous to mention this now OIDs are exposed by default; attached patch (

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-07-09 Thread Dilip Kumar
On Wed, Jul 10, 2019 at 10:15 AM Amit Langote wrote: > > Hi Dilip, > > On Wed, Jul 10, 2019 at 1:29 PM Dilip Kumar wrote: > > On Wed, Jul 10, 2019 at 9:44 AM Dilip Kumar wrote: > > > On Fri, Nov 2, 2018 at 1:34 PM Amit Langote wrote: > > > > Okay, here are two patches: > > > > > > > > 0001 adds

make clean removes excesively

2019-07-09 Thread Kyotaro Horiguchi
I found the following make's behavior is annoying (at dab81b9953). make distclean ./configure make all make -j4 clean all relpath.c:21:10: fatal error: catalog/pg_tablespace_d.h: No such file or directory #include "catalog/pg_tablespace_d.h" (-j is needed, this happnes for me by -j2) Jus

Re: FETCH FIRST clause PERCENT option

2019-07-09 Thread Kyotaro Horiguchi
Hello. At Tue, 9 Jul 2019 21:56:32 -0600, Ryan Lambert wrote in > I did some more testing. I initialized a database with 1 million rows with > indexes and joins to test against and ran pgbench with a few different > settings for % to return. I started with a base query not utilizing the > new

Re: Copy data to DSA area

2019-07-09 Thread Thomas Munro
On Tue, Jun 25, 2019 at 12:53 AM Ideriha, Takeshi wrote: > I've rebased the patch to fit the core code rather than extension. > Regarding shared memory context (ShmContext), I added three > APIs: > - CreatePermShmContext > create "permanent" context located in shared memory > - CreateTempShmCont

Re: range_agg

2019-07-09 Thread David Fetter
On Tue, Jul 09, 2019 at 09:40:59AM -0700, Paul A Jungwirth wrote: > On Tue, Jul 9, 2019 at 8:51 AM David Fetter wrote: > > > - A multirange type is an extra thing you get when you define a range > > > (just like how you get a tstzrange[]). Therefore > > > - I don't need separate commands to ad

Re: POC: Cleaning up orphaned files using undo logs

2019-07-09 Thread Amit Kapila
On Sat, Jul 6, 2019 at 1:47 AM Robert Haas wrote: > > In fact, it seems to me that we shouldn't have any such thing as > "queue entries" at all. The queues should just be pointing to > RollbackHashEntry *, and we should add all the fields there that are > present in any of the "queue entry" struc

Re: pgbench - add \aset to store results of a combined query

2019-07-09 Thread Fabien COELHO
Hello Ibrar, SELECT 1 AS one \; SELECT 2 AS two UNION SELECT 2 \; SELECT 3 AS three \aset will set both "one" and "three", while "two" is not set because there were two rows. It is a kind of more permissive \gset. Are you sure two is not set :)? SELECT 2 AS two UNION SELECT 2; -- only

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-09 Thread Masahiko Sawada
On Tue, Jul 9, 2019 at 9:01 PM Joe Conway wrote: > > On 7/9/19 6:07 AM, Peter Eisentraut wrote: > > On 2019-07-08 18:09, Joe Conway wrote: > >> In my mind, and in practice to a > >> large extent, a postgres tablespace == a unique mount point. > > > > But a critical difference is that in file syste

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-09 Thread Masahiko Sawada
On Tue, Jul 9, 2019 at 10:16 PM Joe Conway wrote: > > On 7/9/19 8:39 AM, Ryan Lambert wrote: > > Hi Thomas, > > > >> CBC mode does require > >> random nonces, other modes may be fine with even sequences as long as > >> the values are not reused. > > > > I disagree that CBC mode requires random non

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-09 Thread Masahiko Sawada
On Wed, Jul 10, 2019 at 11:06 AM Stephen Frost wrote: > > Greetings, > > * Ryan Lambert (r...@rustprooflabs.com) wrote: > > > What I think Tomas is getting at here is that we don't write a page only > > > once. > > > > > A nonce of tableoid+pagenum will only be unique the first time we write > > >

<    1   2