[HACKERS] Refreshing subscription relation state inside a transaction block

Hi, The commit ddd7b22b225ae41d16ceb218b387645cb9becfdc makes table sync workers stop when subscription relation entry is removed. It doesn't work fine inside transaction block. I think we should disallow to use the following subscription DDLs inside a transaction block. Attached patch. * ALTER S

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-06-09 22:28:00 +0200, Petr Jelinek wrote: > On 09/06/17 03:20, Petr Jelinek wrote: > > On 09/06/17 01:06, Andres Freund wrote: > >> Hi, > >> > >> On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: > >>> One thing I don't like is the GetLastLocalTransactionId() that I had to > >>> add because

[HACKERS] Detection of IPC::Run presence in SSL TAP tests

Hi all, 001_ssltests.pl in src/test/ssl/ includes the following to skip all tests should IPC::Run be not available: # Like TestLib.pm, we use IPC::Run BEGIN { eval { require IPC::Run; import IPC::Run qw(run start); 1; } or do

Re: [HACKERS] logical replication busy-waiting on a lock

On 2017-06-13 00:32:40 -0700, Andres Freund wrote: > On 2017-06-09 22:28:00 +0200, Petr Jelinek wrote: > > On 09/06/17 03:20, Petr Jelinek wrote: > > > On 09/06/17 01:06, Andres Freund wrote: > > >> Hi, > > >> > > >> On 2017-06-03 04:50:00 +0200, Petr Jelinek wrote: > > >>> One thing I don't like i

Re: [HACKERS] pg_receivewal and messages printed in non-verbose mode

On 13 June 2017 at 14:33, Michael Paquier wrote: > Hi all, > > I have noticed that the following messages can show up from > pg_receivewal even if the verbose mode is not used: > if (prevtimeline != 0 && prevtimeline != timeline) > fprintf(stderr, _("%s: switched to timelin

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

On Tue, Jun 13, 2017 at 6:40 PM, Noah Misch wrote: > On Mon, Jun 12, 2017 at 04:04:23PM +1200, Thomas Munro wrote: >> On Sun, Jun 11, 2017 at 11:11 PM, Thomas Munro >> wrote: >> > On Sun, Jun 11, 2017 at 6:25 PM, Noah Misch wrote: >> >> This fourth point is not necessarily a defect: I wonder if

Re: [HACKERS] Refreshing subscription relation state inside a transaction block

On 13/06/17 09:06, Masahiko Sawada wrote: > Hi, > > The commit ddd7b22b225ae41d16ceb218b387645cb9becfdc makes table sync > workers stop when subscription relation entry is removed. It doesn't > work fine inside transaction block. I think we should disallow to use > the following subscription DDLs

[HACKERS] fix possible optimizations in ATExecAttachPartition()

Hi, I was doing some testing for my default partitioning work, and I realized that there seem to be a some optimization possible (and I think we should really have this optimization) for logic around handling the "key IS NOT NULL" constraints followed by predicate_implied_by() call. 1. Consider,

Re: [HACKERS] fix possible optimizations in ATExecAttachPartition()

On 2017/06/13 18:08, Jeevan Ladhe wrote: > Hi, > > I was doing some testing for my default partitioning work, and I realized > that > there seem to be a some optimization possible (and I think we should really > have > this optimization) for logic around handling the "key IS NOT NULL" > constraint

Re: [HACKERS] UPDATE of partition key

While rebasing my patch for the below recent commit, I realized that a similar issue exists for the uptate-tuple-routing patch as well : commit 78a030a441966d91bc7e932ef84da39c3ea7d970 Author: Tom Lane Date: Mon Jun 12 23:29:44 2017 -0400 Fix confusion about number of subplans in partition

Re: [HACKERS] Broken hint bits (freeze)

On Mon, Jun 12, 2017 at 9:01 PM, Vladimir Borodin wrote: > > 12 июня 2017 г., в 13:19, Amit Kapila написал(а): > > On Sun, Jun 11, 2017 at 11:59 PM, Vladimir Borodin wrote: > > > 8 июня 2017 г., в 17:03, Amit Kapila написал(а): > > On Thu, Jun 8, 2017 at 6:49 PM, Dmitriy Sarafannikov > wrote:

[HACKERS] Typo in BRIN documentation

Hi, I just found this typo while doing french translation, patch attached. Regards. -- Julien Rouhaud http://dalibo.com - http://dalibo.org diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml index ad11109775..8dcc29925b 100644 --- a/doc/src/sgml/brin.sgml +++ b/doc/src/sgml/brin.sgml @

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

On Mon, Jun 12, 2017 at 5:25 PM, Andres Freund wrote: > Even there I don't think that's a sane assumption *for the future*. We > just need a slight change in the rules about when a toast table is needed > - and that stuff seriously need overhauling - and it doesn't work > anymore. The problem is

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

On Mon, Jun 12, 2017 at 12:04 AM, Thomas Munro wrote: > Here are a couple of ways forward that I can see: > > 1. Figure out how to get the QueryEnvironment through more of these > stack frames (possibly inside other objects), so that > set_namedtuplestore_size_estimates can look up enrtuples by e

[HACKERS] GSoC 2017 weekly progress reports (week 2)

Project: Explicitly support predicate locks in index AMs besides b-tree Hi, During this week, I mostly worked on testing to verify my code and on debugging to solve some issues I was having. I have specifically created two tests. The first test is about verifying serialization failure when there

Re: [HACKERS] WIP: Data at rest encryption

On 6/12/17 17:11, Ants Aasma wrote: > I'm curious if the community thinks this is a feature worth having? > Even considering that security experts would classify this kind of > encryption as a checkbox feature. File system encryption already exists and is well-tested. I don't see any big advantag

Re: [HACKERS] ICU support on Windows

On 6/12/17 14:03, Ashutosh Sharma wrote: >> I noticed that this only works if you use the "Win32" download of ICU, >> because the "Win64" download uses "lib64" paths. I'm not sure what the >> impact of this is in practice. > > Yes, that's right, Win64 download uses lib64 path and in my case i had

Re: [HACKERS] WIP: Data at rest encryption

Ants, all, * Ants Aasma (ants.aa...@eesti.ee) wrote: > Yes, the plan is to pick it up again, Real Soon Now(tm). There are a > couple of loose ends for stuff that should be encrypted, but in the > current state of the patch aren't yet (from the top of my head, > logical decoding and pg_stat_stateme

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

Robert Haas writes: > The problem is that if a relfilenode ever gets assigned by > GetNewRelFileNode() during a binary-upgrade restore, that OID may turn > out to be used by some other object later in the dump. And then > you're dead, because the dump restore will fail later on complaining > abou

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

I reviewed and tested 0001-Dependency-between-partitioned-table-and-partition_v1.patch It applies cleanly on master and make check passes. Following are few comments: >/* > * Drop the dependency created by StoreCatalogInheritance1 (CREATE TABLE > * INHERITS/ALTER TABLE INHERIT -- refclassid will

Re: [HACKERS] pgrowlocks relkind check

On 6/12/17 21:10, Amit Langote wrote: > On 2017/06/13 0:29, Peter Eisentraut wrote: >> On 4/24/17 21:22, Amit Langote wrote: > create extension pgrowlocks; > create view one as select 1; > select pgrowlocks('one'); > -- ERROR: could not open file "base/68730/68748": No such file or

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

Robert Haas writes: > Perhaps this is a silly question, but I don't particularly see what's > wrong with: > 3. Do nothing. Well, the fundamental problem is that the RTE is a lousy place to keep rowcount estimates. That breaks assorted desirable properties like querytrees being readonly to plann

Re: [HACKERS] Detection of IPC::Run presence in SSL TAP tests

Michael Paquier writes: > 001_ssltests.pl in src/test/ssl/ includes the following to skip all > tests should IPC::Run be not available: > ... > In all the other tests or modules we don't bother about such a thing > as prove_check only works if --enable-tap-test is used, and we get a > hard failure

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

On Mon, Jun 12, 2017 at 4:09 AM, Amit Langote wrote: > On 2017/06/09 20:49, Ashutosh Bapat wrote: >> May be we should pass a flag to predicate_implied_by() to handle NULL >> behaviour for CHECK constraints. Partitioning has shown that it needs >> to use predicate_implied_by() for comparing constra

Re: [HACKERS] WIP: Data at rest encryption

On 6/13/17 09:24, Stephen Frost wrote: > but there are > use-cases where it'd be really nice to be able to have PG doing the > encryption instead of the filesystem because then you can do things like > backup the database, copy it somewhere else directly, and then restore > it using the regular PG

Re: [HACKERS] tablesync.c - comment improvements

On 6/10/17 04:52, Erik Rijkers wrote: > tablesync.c - comment improvements Committed, thanks! -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Fix a typo in shm_mq.c

On 6/12/17 21:32, Masahiko Sawada wrote: > Attached the patch for $subject. > > s/Whem/When/ committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] WIP: Data at rest encryption

Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > I wonder what the proper extent of "encryption at rest" should be. If > you encrypt just on a file or block level, then someone looking at the > data directory or a backup can still learn a number of things about the > number o

Re: [HACKERS] macOS Sierra & System Integrity Protection

On 6/12/17 23:38, Tom Lane wrote: > https://www.postgresql.org/message-id/26098.1446697...@sss.pgh.pa.us > >> My main purpose in writing this email is to pass along what I learned >> in the hopes of sparing somebody else some trouble, but perhaps there >> is a way to modify our regression test set

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

On Tue, Jun 13, 2017 at 9:37 AM, Tom Lane wrote: > In the long run, it would certainly be cleaner if pg_upgrade dropped > the force-the-relfilenode-assignment approach and instead remapped > relfilenodes from old cluster to new. But I think it's just for > cleanliness rather to fix any live or fo

Re: [HACKERS] macOS Sierra & System Integrity Protection

Simon Riggs writes: > On 13 June 2017 at 04:25, Robert Haas wrote: >> 'make check' was failing: 'psql' repeatedly died with an abort >> trap. Binaries worked fine when I ran them from the command line >> (sometimes with DYLD_LIBRARY_PATH, if needed) but when run via >> pg_regress, nothing worked

[HACKERS] outfuncs.c utility statement support

While debugging some other stuff, I was wondering to what extent node types supporting utility statements should be supported in outfuncs.c. Running with --debug-print-parse=on, executing create table test1 (a int, b text); gives output that is truncated somewhere in the middle (possibly a null b

Re: [HACKERS] macOS Sierra & System Integrity Protection

On 13 June 2017 at 04:25, Robert Haas wrote: > I have a new MacBook Pro running Sierra. Congratulations. > 'make check' was failing: 'psql' repeatedly died with an abort > trap. Binaries worked fine when I ran them from the command line > (sometimes with DYLD_LIBRARY_PATH, if needed) but when

Re: [HACKERS] WIP: Data at rest encryption

On Mon, Jun 12, 2017 at 5:11 PM, Ants Aasma wrote: > Fundamentally there doesn't seem to be a big benefit of implementing > the encryption at PostgreSQL level instead of the filesystem. The > patch doesn't take any real advantage from the higher level knowledge > of the system, nor do I see much p

Re: [HACKERS] Typo in BRIN documentation

On 6/13/17 07:53, Julien Rouhaud wrote: > I just found this typo while doing french translation, patch attached. fixed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] outfuncs.c utility statement support

Peter Eisentraut writes: > While debugging some other stuff, I was wondering to what extent node > types supporting utility statements should be supported in outfuncs.c. We've largely not tried too hard in that department. From a debugging standpoint it'd be useful to have more complete coverage

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

On Tue, Jun 13, 2017 at 10:05 AM, Tom Lane wrote: > Robert Haas writes: >> Perhaps this is a silly question, but I don't particularly see what's >> wrong with: > >> 3. Do nothing. > > Well, the fundamental problem is that the RTE is a lousy place to keep > rowcount estimates. That breaks assorte

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 11:35:03AM -0400, Robert Haas wrote: > I anticipate that one of the trickier problems here will be handling > encryption of the write-ahead log. Suppose you encrypt WAL a block at > a time. In the current system, once you've written and flushed a > block, you can consider

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

Robert Haas writes: > On Tue, Jun 13, 2017 at 10:05 AM, Tom Lane wrote: >> Well, the fundamental problem is that the RTE is a lousy place to keep >> rowcount estimates. That breaks assorted desirable properties like >> querytrees being readonly to planning/execution (not that we don't >> end up

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

On Tue, Jun 13, 2017 at 9:44 AM, Rahila Syed wrote: > I have added tests to the > 0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please > find attached the v2 patch. Thanks. Committed. I don't think the 0002 patch is an improvement - sure, it keeps those things in sync, but i

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

On Tue, Jun 13, 2017 at 11:53 AM, Tom Lane wrote: >> How does it break those properties? I don't think enrtuples is being >> modified by planning or execution as things stand. > > But it needs to be changeable, unless you like the proposition that we > can never replan a query inside a trigger on

Re: [HACKERS] Refreshing subscription relation state inside a transaction block

On Tue, Jun 13, 2017 at 4:53 PM, Petr Jelinek wrote: > On 13/06/17 09:06, Masahiko Sawada wrote: >> Hi, >> >> The commit ddd7b22b225ae41d16ceb218b387645cb9becfdc makes table sync >> workers stop when subscription relation entry is removed. It doesn't >> work fine inside transaction block. I think

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

Robert Haas writes: > On Tue, Jun 13, 2017 at 11:53 AM, Tom Lane wrote: >> But it needs to be changeable, unless you like the proposition that we >> can never replan a query inside a trigger on the basis of new information >> about how big the transition table is. Even if you're okay with that >

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 11:04:21AM -0400, Stephen Frost wrote: > > Also, in the use case you describe, if you use pg_basebackup to make a > > direct encrypted copy of a data directory, I think that would mean you'd > > have to keep using the same key for all copies. > > That's true, but that might

Re: [HACKERS] Typo in BRIN documentation

On Tue, Jun 13, 2017 at 11:29:30AM -0400, Peter Eisentraut wrote: > On 6/13/17 07:53, Julien Rouhaud wrote: > > I just found this typo while doing french translation, patch attached. > > fixed > Thanks ! -- Julien Rouhaud http://dalibo.com - http://dalibo.org -- Sent via pgsql-hackers mailing

Re: [HACKERS] memory fields from getrusage()

On Sat, Jun 10, 2017 at 9:31 PM, Tom Lane wrote: > We already do call getrusage(). The point of that comment is that the > contents of the resulting struct rusage are not very well standardized. > POSIX says only > > The header defines the rusage structure that includes > at least the fo

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

On Tue, Jun 13, 2017 at 12:04 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jun 13, 2017 at 11:53 AM, Tom Lane wrote: >>> But it needs to be changeable, unless you like the proposition that we >>> can never replan a query inside a trigger on the basis of new information >>> about how big

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

Piotr Stefaniak writes: >> There's also the portability issues: __FBSDID() and bcopy() and >> [and err.h]. > I think that's fixed as well. I just finished some preliminary portability testing and things look much improved. The Makefile is still BSD-ish of course, but I think we'll just agree t

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 11:04:21AM -0400, Stephen Frost wrote: > > > Also, in the use case you describe, if you use pg_basebackup to make a > > > direct encrypted copy of a data directory, I think that would mean you'd > > > have to keep using the

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 12:23 PM, Stephen Frost wrote: > Key management is an entirely independent discussion from this and the > proposal from Ants, as I understand it, is that the key would *not* be > in the database but could be anywhere that a shell command could get it > from, including possi

Re: [HACKERS] Refreshing subscription relation state inside a transaction block

On Wed, Jun 14, 2017 at 1:02 AM, Masahiko Sawada wrote: > On Tue, Jun 13, 2017 at 4:53 PM, Petr Jelinek > wrote: >> On 13/06/17 09:06, Masahiko Sawada wrote: >>> Hi, >>> >>> The commit ddd7b22b225ae41d16ceb218b387645cb9becfdc makes table sync >>> workers stop when subscription relation entry is r

Re: [HACKERS] remove unnecessary flag has_null from PartitionBoundInfoData

On Mon, Jun 12, 2017 at 9:03 AM, Ashutosh Bapat wrote: > On Mon, Jun 12, 2017 at 3:50 PM, amul sul wrote: >> On Wed, May 17, 2017 at 10:22 PM, Robert Haas wrote: >> [...] >>> I committed this with fixes for those issues, plus I renamed the macro >>> to partition_bound_accepts_nulls, which I thin

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 12:23:01PM -0400, Stephen Frost wrote: > > As I understand it, having encryption in the database means the key is > > stored in the database, while having encryption in the file system means > > the key is stored in the operating system somewhere. > > Key management is an

Re: [HACKERS] WIP: Data at rest encryption

On 06/13/2017 09:28 AM, Robert Haas wrote: > On Tue, Jun 13, 2017 at 12:23 PM, Stephen Frost wrote: >> Key management is an entirely independent discussion from this and the >> proposal from Ants, as I understand it, is that the key would *not* be >> in the database but could be anywhere that a sh

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: > > That way, if the user wants to store the key in an unencrypted text > > file, they can set the encryption_key_command = 'cat /not/very/secure' > > and call it a day. If they want to prompt the user on the console or > > request the ke

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 12:23:01PM -0400, Stephen Frost wrote: > > > Of course, if the > > > key stored in the database is visible to someone using the operating > > > system, we really haven't added much/any security --- I guess my point > > > is

Re: [HACKERS] WIP: Data at rest encryption

Bruce, Joe, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: > > > That way, if the user wants to store the key in an unencrypted text > > > file, they can set the encryption_key_command = 'cat /not/very/secure' > > > and call it a day. If th

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

On Tue, Jun 13, 2017 at 11:14:02AM -0400, Robert Haas wrote: > Also, I think that if we did it that way, it would be significantly > harder to debug. Right now, if something goes boom, you can look at > the old and new clusters and figure out what doesn't match, but if > pg_upgrade renumbered ever

Re: [HACKERS] WIP: Data at rest encryption

On 06/13/2017 10:05 AM, Stephen Frost wrote: > Bruce, Joe, > > * Bruce Momjian (br...@momjian.us) wrote: >> On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: >> > > That way, if the user wants to store the key in an unencrypted text >> > > file, they can set the encryption_key_command =

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 01:01:32PM -0400, Stephen Frost wrote: > > Well, usually the symetric key is stored using RSA and a symetric > > cipher is used to encrypt/decrypt the data. I was thinking of a case > > where you encrypt a row using a symetric key, then store RSA-encrypted > > versions of t

Re: [HACKERS] Why are we restricting exported snapshots in subtransactions?

On Mon, Jun 12, 2017 at 11:04 PM, Andres Freund wrote: > ExportSnapshot() has, right at the beginning, the following block: > > /* > * We cannot export a snapshot from a subtransaction because there's no > * easy way for importers to verify that the same subtransaction is still >

[HACKERS] Announcing Release 5 of the PostgreSQL Buildfarm Client

Release 5 of the PostgreSQL Buildfarm Client has been released and can be downloaded from In a similar move to PostgreSQL version numbering, with this release we move to a one part numbering system. In addition to a number

Re: [HACKERS] WIP: Data at rest encryption

Joe, * Joe Conway (m...@joeconway.com) wrote: > Except shell escaping issues, etc, etc That's not an issue- we're talking about reading the stdout of some other process, there's no shell escaping that has to be done there. > > Let us, please, stop stressing over the right way to do key managemen

Re: [HACKERS] Why are we restricting exported snapshots in subtransactions?

On 2017-06-13 13:15:57 -0400, Robert Haas wrote: > On Mon, Jun 12, 2017 at 11:04 PM, Andres Freund wrote: > > ExportSnapshot() has, right at the beginning, the following block: > > > > /* > > * We cannot export a snapshot from a subtransaction because there's no > > * easy way for im

Re: [HACKERS] v10beta pg_catalog diagrams

On Mon, Jun 12, 2017 at 04:07:35PM -0400, Peter Eisentraut wrote: > On 6/12/17 11:28, Neil Anderson wrote: > > I'm cross posting from general. I did some work to diagram the > > relationships in pg_catalog for v10. I would like to add it to the > > developer FAQ here > > https://wiki.postgresql.

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:01:32PM -0400, Stephen Frost wrote: > > > Well, usually the symetric key is stored using RSA and a symetric > > > cipher is used to encrypt/decrypt the data. I was thinking of a case > > > where you encrypt a row using

Re: [HACKERS] WIP: Data at rest encryption

On 06/13/2017 10:20 AM, Stephen Frost wrote: > * Joe Conway (m...@joeconway.com) wrote: >> Except shell escaping issues, etc, etc > > That's not an issue- we're talking about reading the stdout of some > other process, there's no shell escaping that has to be done there. It could be an issue depe

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

On 13 June 2017 at 05:50, Ashutosh Bapat wrote: > On Tue, Jun 13, 2017 at 12:03 AM, Dean Rasheed > wrote: >> Barring objections, I'll push my original patch and work up patches >> for the other couple of issues I found. > > No objections, the patch is good to go as is. Sorry for high-jacking > t

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 01:25:00PM -0400, Stephen Frost wrote: > > I think the big win of Postgres doing the encryption is that the > > user-visible file system is no longer a target (assuming OS permissions > > are bypassed), while for file system encryption it is the storage device > > that is en

Re: [HACKERS] WIP: Data at rest encryption

Joe, * Joe Conway (m...@joeconway.com) wrote: > On 06/13/2017 10:20 AM, Stephen Frost wrote: > > * Joe Conway (m...@joeconway.com) wrote: > >> Except shell escaping issues, etc, etc > > > > That's not an issue- we're talking about reading the stdout of some > > other process, there's no shell esc

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 01:44:51PM -0400, Stephen Frost wrote: > Just to be clear, I don't have any issue with discussing the idea that > we want to get to a point where we can work with multiple keys and > encrypt different tables with different keys (or not encrypt certain > tables, et al) with t

Re: [HACKERS] GSoC 2017 weekly progress reports (week 2)

2017-06-13 18:00 GMT+05:00 Shubham Barai : > > Project: Explicitly support predicate locks in index AMs besides b-tree > Hi, Shubham Good job! So, in current HEAD test predicate_gist_2.spec generate false positives, but with your patch, it does not? I'd suggest keeping spec tests with your code in

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:44:51PM -0400, Stephen Frost wrote: > > Just to be clear, I don't have any issue with discussing the idea that > > we want to get to a point where we can work with multiple keys and > > encrypt different tables with diff

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:25:00PM -0400, Stephen Frost wrote: > > > I think the big win of Postgres doing the encryption is that the > > > user-visible file system is no longer a target (assuming OS permissions > > > are bypassed), while for file

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 02:23:39PM -0400, Stephen Frost wrote: > I'm not trying to shut down discussion, I'm simply pointing out where > this feature will be helpful and where it won't be. If there's a way to > make it better and able to address an attack where the OS permission > system is bypass

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 02:23:39PM -0400, Stephen Frost wrote: > > I'm not trying to shut down discussion, I'm simply pointing out where > > this feature will be helpful and where it won't be. If there's a way to > > make it better and able to ad

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > It's good to discuss what the feature would bring and what cases it > doesn't cover, as well as discussing how it can be designed to make sure > that later improvements are able to be done without having to change it > around. I do t

Re: [HACKERS] v10beta pg_catalog diagrams

On 2017-06-13 1:22 PM, Bruce Momjian wrote: On Mon, Jun 12, 2017 at 04:07:35PM -0400, Peter Eisentraut wrote: On 6/12/17 11:28, Neil Anderson wrote: I'm cross posting from general. I did some work to diagram the relationships in pg_catalog for v10. I would like to add it to the developer FAQ he

Re: [HACKERS] WIP: Data at rest encryption

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > It's good to discuss what the feature would bring and what cases it > > doesn't cover, as well as discussing how it can be designed to make sure > > that later improvements are abl

Re: [HACKERS] Broken hint bits (freeze)

On Mon, Jun 12, 2017 at 06:31:11PM +0300, Vladimir Borodin wrote: > What about the following sequence? > > 1. Run pg_upgrade on master, > 2. Start it in single-user mode and stop (to get right wal_level in > pg_control), > 3. Copy pg_control somewhere, > 4. Start master, run analyze and stop. > 5.

Re: [HACKERS] WIP: Data at rest encryption

On 6/13/17 15:20, Stephen Frost wrote: > No, the benefit is that the database administrator can configure it and > set it up and not have to get an OS-level administrator involved. There > may also be other reasons why filesystem-level encryption is difficult > to set up or use in a certain enviro

Re: [HACKERS] WIP: Data at rest encryption

On 6/13/17 15:20, Stephen Frost wrote: > For example, you could simply do: > > cp -a /path/to/PG /mnt/usb > > and you're done. If you're using filesystem level encryption then you'd > have to re-encrypt the data, using something like: > > tar -cf - /path/to/PG | openssl -key private.key > > /

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

On 6/13/17 02:33, Noah Misch wrote: >> Steps to reproduce - >> X cluster -> create 100 tables , publish all tables (create publication pub >> for all tables); >> Y Cluster -> create 100 tables ,create subscription(create subscription sub >> connection 'user=centos host=localhost' publication pub;

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > > It's good to discuss what the feature would bring and what cases it > > > doesn't cover, as well as discuss

Re: [HACKERS] WIP: Data at rest encryption

On 6/13/17 15:51, Bruce Momjian wrote: > Isn't the leakage controlled by OS permissions, so is it really leakage, > i.e., if you can see the leakage, you probably have bypassed the OS > permissions and see the key and data anyway. One scenario (among many) is when you're done with the disk. If th

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 04:08:29PM -0400, Peter Eisentraut wrote: > On 6/13/17 15:51, Bruce Momjian wrote: > > Isn't the leakage controlled by OS permissions, so is it really leakage, > > i.e., if you can see the leakage, you probably have bypassed the OS > > permissions and see the key and data an

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

I've now done a round of comparisons of results of our old indent with your current version. There's still one serious bug in the latter: it continues to misformat enum typedefs, for instance *** PG_FUNCTION_INFO_V1(pg_prewarm); *** 33,40 typedef enum { PREWARM_PREFET

Re: [HACKERS] GSoC 2017 weekly progress reports (week 2)

On Tue, Jun 13, 2017 at 1:02 PM, Andrew Borodin wrote: > 2017-06-13 18:00 GMT+05:00 Shubham Barai : > Good job! +1! :-) > So, in current HEAD test predicate_gist_2.spec generate false > positives, but with your patch, it does not? Keep in mind, that false positives do not break *correctness*

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

On Tue, Jun 13, 2017 at 10:24 AM, Robert Haas wrote: > I think that's going to come as an unpleasant surprise to more than > one user. I'm not sure exactly how we need to restructure things here > so that this works properly, and maybe modifying > predicate_implied_by() isn't the right thing at a

Re: [HACKERS] v10beta pg_catalog diagrams

On Tue, Jun 13, 2017 at 3:14 PM, Neil Anderson wrote: > There were a few relationships that I couldn't capture. Like where in > pg_extension extconfig is an array of oids that refer to pg_class or where > pg_depends could refer to basically any other system catalog, but it's > mostly there and has

Re: [HACKERS] fix possible optimizations in ATExecAttachPartition()

On Tue, Jun 13, 2017 at 5:22 AM, Amit Langote wrote: > Yeah, I was thinking the same while writing the patch posted on the thread > "A bug in mapping attributes in ATExecAttachPartition()" [1]. That patch > adds the break you mention in 2, but didn't do anything about point 1. > > In any case, +1

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

On 2017-06-13 18:22, Tom Lane wrote: > The Makefile is still BSD-ish of course, but I think > we'll just agree to disagree there. For compiling indent under Linux I use bmake(1). I have no problem with including a Makefile for GNU Make in my repository. As I understand it, there will be a copy of

Re: [HACKERS] WIP: Data at rest encryption

On Tue, Jun 13, 2017 at 10:28:14AM -0400, Peter Eisentraut wrote: > On 6/13/17 09:24, Stephen Frost wrote: > > but there are use-cases where it'd be really nice to be able to > > have PG doing the encryption instead of the filesystem because > > then you can do things like backup the database, copy

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

On Wed, Jun 14, 2017 at 4:22 AM, Robert Haas wrote: > I'm just trying to understand your comments so that I can have an > intelligent opinion about what we should do from here. Given that the > replan wouldn't happen anyway, there seems to be no reason to tinker > with the location of enrtuples f

Re: [HACKERS] v10beta pg_catalog diagrams

Hi, On 2017-06-12 11:28:39 -0400, Neil Anderson wrote: > I'm cross posting from general. I did some work to diagram the relationships > in pg_catalog for v10. I would like to add it to the developer FAQ here > https://wiki.postgresql.org/wiki/Developer_FAQ#Is_there_a_diagram_of_the_system_catalog

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

On Tue, Jun 13, 2017 at 05:00:31PM -0400, Tom Lane wrote: > Anyway, it is now time to fish or cut bait. I don't think we can wait > much longer to decide whether we're going to adopt this new indent > version for PG 10. I think we should. The floor is open for votes. Works for me. -- Bruce

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

On 2017-06-13 22:23, Tom Lane wrote: > I could not find any places where reverting this change made the > results worse, so I'm unclear on why you made it. I must admit I'm a bit confused about why it's not fixed yet, but I'll have to analyze that later this week. But the idea was to convince inde

Re: [HACKERS] fix possible optimizations in ATExecAttachPartition()

On Wed, Jun 14, 2017 at 2:12 AM, Robert Haas wrote: > On Tue, Jun 13, 2017 at 5:22 AM, Amit Langote > wrote: > > Yeah, I was thinking the same while writing the patch posted on the > thread > > "A bug in mapping attributes in ATExecAttachPartition()" [1]. That patch > > adds the break you menti

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

Piotr Stefaniak writes: > On 2017-06-13 22:23, Tom Lane wrote: >> I could not find any places where reverting this change made the >> results worse, so I'm unclear on why you made it. > I must admit I'm a bit confused about why it's not fixed yet, but I'll > have to analyze that later this week.

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

Robert Haas writes: > OK, I think I see the problem here. predicate_implied_by() and > predicate_refuted_by() differ in what they assume about the predicate > evaluating to NULL, but both of them assume that if the clause > evaluates to NULL, that's equivalent to false. So there's actually no >

  1   2   >