Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Jeff Davis
On Sat, 2019-08-10 at 10:24 +0800, Craig Ringer wrote: > Before we go too far along with this, lets look at how other > established protocols do things and the flaws that've been discovered > in their approaches. If this isn't done with extreme care then > there's a large risk of negating the benef

Re: POC: converting Lists into arrays

2019-08-09 Thread David Rowley
On Sat, 10 Aug 2019 at 09:03, Tom Lane wrote: > > David Rowley writes: > > On Fri, 9 Aug 2019 at 09:55, Tom Lane wrote: > >> I still have hopes for getting rid of es_range_table_array though, > >> and will look at that tomorrow or so. > > > Yes, please. I've measured that to be quite an overhead

Re: Locale support

2019-08-09 Thread Thomas Munro
On Sat, Aug 10, 2019 at 11:50 AM Thomas Munro wrote: > On Fri, Aug 9, 2019 at 6:15 PM Yonatan Misgan wrote: > > Can I implement it as a locale support? When the user want to change the > > lc _time = am_ET(Amharic Ethiopia ) the date and time representation of the > > database systems be in Et

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Bruce Momjian
On Thu, Aug 8, 2019 at 10:17:53PM -0400, Sehrope Sarkuni wrote: > On Thu, Aug 8, 2019 at 2:16 PM Bruce Momjian wrote: > > On Wed, Aug  7, 2019 at 08:56:18AM -0400, Sehrope Sarkuni wrote: > > Simplest approach for derived keys would be to use immutable attributes > of the > > WAL

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Bruce Momjian
On Fri, Aug 9, 2019 at 05:01:47PM +0200, Tomas Vondra wrote: > I know there were proposals to keep it encrypted in shared buffers, but > I'm not sure that's what we'll end up doing (I have not followed the > recent discussion all that closely, though). There is no plan to encrypt shared buffers.

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Craig Ringer
On Fri, 9 Aug 2019 at 11:00, Michael Paquier wrote: > On Thu, Aug 08, 2019 at 03:38:20PM -0700, Jeff Davis wrote: > > Libpq doesn't have a way to control which password protocols are used. > > For example, the client might expect the server to be using SCRAM, but > > it actually ends up using pla

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Bruce Momjian
On Thu, Aug 8, 2019 at 10:34:26PM -0400, Sehrope Sarkuni wrote: > On Thu, Aug 8, 2019 at 6:31 PM Stephen Frost wrote: > > Strictly speaking, that isn't actually crash recovery, it's physical > replication / HA, and while those are certainly nice to have it's no > guarantee that they'

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Bruce Momjian
On Fri, Aug 9, 2019 at 11:51:23PM +0900, Masahiko Sawada wrote: > I agree that cluster-wide is more simpler but I'm not sure that it > meets real needs from users. One example is re-encryption; when the > key leakage happens, in cluster-wide encryption we end up with doing > re-encrypt whole datab

Re: Global temporary tables

2019-08-09 Thread Craig Ringer
On Fri, 9 Aug 2019 at 22:07, Konstantin Knizhnik wrote: > > > Ok, here it is: global_private_temp-1.patch > Fantastic. I'll put that high on my queue. I'd love to see something like this get in. Doubly so if it brings us closer to being able to use temp tables on physical read replicas, thoug

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Stephen Frost
Greetings, * Jeff Davis (pg...@j-davis.com) wrote: > On Sat, 2019-08-10 at 00:17 +0300, Heikki Linnakangas wrote: > > auth_methods = 'MITM, -password, -md5' > > Keep in mind this is client configuration, so something reasonable in > postgresql.conf might not be so reasonable in the form: Yeah, t

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Jeff Davis
On Sat, 2019-08-10 at 00:17 +0300, Heikki Linnakangas wrote: > This is a multi-dimensional problem. "channel_binding=require" is > one > way to prevent MITM attacks, but sslmode=verify-ca is another. (Does > Kerberos also prevent MITM?) Or you might want to enable plaintext > passwords over SSL,

Re: Locale support

2019-08-09 Thread Thomas Munro
On Fri, Aug 9, 2019 at 6:15 PM Yonatan Misgan wrote: > Can I implement it as a locale support? When the user want to change the lc > _time = am_ET(Amharic Ethiopia ) the date and time representation of the > database systems be in Ethiopian calendar. Hi Yonatan, I'm not an expert in this stuf

Shrinking tuplesort.c's SortTuple struct (Was: More ideas for speeding up sorting)

2019-08-09 Thread Peter Geoghegan
On Fri, Sep 9, 2016 at 6:14 AM Heikki Linnakangas wrote: > 1. SortTuple.tupindex is not used when the sort fits in memory. If we > also got rid of the isnull1 flag, we could shrink SortTuple from 24 > bytes to 16 bytes (on 64-bit systems). That would allow you to pack more > tuples into memory, wh

Re: POC: converting Lists into arrays

2019-08-09 Thread Alvaro Herrera
On 2019-Aug-09, Tom Lane wrote: > I poked at this, and attached is a patch, but again I'm not seeing > that there's any real performance-based argument for it. So far > as I can tell, if we've got a lot of RTEs in an executable plan, > the bulk of the startup time is going into lock (re) acquisit

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Jeff Davis
On Fri, 2019-08-09 at 16:27 -0400, Jonathan S. Katz wrote: > Seems to be a lot to configure. I'm more of a fan of the previous > method; it'd work nicely with how we've presently defined things and > should be easy to put into a DSN/URI/env variable. Proposals on the table: 1. Hierarchical semant

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Heikki Linnakangas
On 09/08/2019 23:27, Jonathan S. Katz wrote: On 8/9/19 11:51 AM, Jeff Davis wrote: On Fri, 2019-08-09 at 09:28 -0400, Stephen Frost wrote: Having an 'any' option, as mentioned before, could be an alternative though. ... I agree with the point that there isn't any guarantee that it'll always

Re: POC: converting Lists into arrays

2019-08-09 Thread Tom Lane
David Rowley writes: > On Fri, 9 Aug 2019 at 09:55, Tom Lane wrote: >> I still have hopes for getting rid of es_range_table_array though, >> and will look at that tomorrow or so. > Yes, please. I've measured that to be quite an overhead with large > partitioning setups. However, that was with so

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Jonathan S. Katz
On 8/9/19 11:51 AM, Jeff Davis wrote: > On Fri, 2019-08-09 at 09:28 -0400, Stephen Frost wrote: >> Having an 'any' option, as mentioned before, could be an alternative >> though. > > ... > >> I agree with the point that there isn't any guarantee that it'll >> always >> be clear-cut as to which of

Re: Some memory not freed at the exit of RelationBuildPartitionDesc()

2019-08-09 Thread David Fetter
On Thu, Aug 08, 2019 at 05:42:21PM +0900, Amit Langote wrote: > On Thu, Aug 8, 2019 at 5:33 PM amul sul wrote: > > On Thu, Aug 8, 2019 at 1:27 PM Amit Langote wrote: > > >> Thanks for the patch. This was discussed recently in the "hyrax vs. > >> RelationBuildPartitionDesc()" thread [1] and I th

Re: block-level incremental backup

2019-08-09 Thread Jeevan Ladhe
Hi Robert, On Fri, Aug 9, 2019 at 6:40 PM Robert Haas wrote: > On Thu, Aug 8, 2019 at 8:37 PM Jeevan Ladhe > wrote: > > + if (!XLogRecPtrIsInvalid(previous_lsn)) > > + appendStringInfo(labelfile, "PREVIOUS WAL LOCATION: %X/%X\n", > > +(uint32) (previo

Re: POC: Cleaning up orphaned files using undo logs

2019-08-09 Thread Robert Haas
On Wed, Aug 7, 2019 at 6:57 AM Heikki Linnakangas wrote: > Yeah, that's also a problem with complicated WAL record types. Hopefully > the complex cases are an exception, not the norm. A complex case is > unlikely to fit any pre-defined set of fields anyway. (We could look at > how e.g. protobuf wo

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Jeff Davis
On Fri, 2019-08-09 at 09:28 -0400, Stephen Frost wrote: > Having an 'any' option, as mentioned before, could be an alternative > though. ... > I agree with the point that there isn't any guarantee that it'll > always > be clear-cut as to which of two methods is "better". > > From a user perspect

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Tomas Vondra
On Fri, Aug 09, 2019 at 11:51:23PM +0900, Masahiko Sawada wrote: On Fri, Aug 9, 2019 at 10:25 AM Bruce Momjian wrote: On Thu, Aug 8, 2019 at 06:31:42PM -0400, Stephen Frost wrote: > > >Crash recovery doesn't happen "all the time" and neither does vacuum > > >freeze, and autovacuum processes a

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Tomas Vondra
On Thu, Aug 08, 2019 at 06:31:42PM -0400, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On Thu, Aug 08, 2019 at 03:07:59PM -0400, Stephen Frost wrote: >* Bruce Momjian (br...@momjian.us) wrote: >>On Tue, Jul 9, 2019 at 11:09:01AM -0400, Bruce Momjian wrot

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-09 Thread Masahiko Sawada
On Fri, Aug 9, 2019 at 10:25 AM Bruce Momjian wrote: > > On Thu, Aug 8, 2019 at 06:31:42PM -0400, Stephen Frost wrote: > > > >Crash recovery doesn't happen "all the time" and neither does vacuum > > > >freeze, and autovacuum processes are independent of individual client > > > >backends- we don't

Re: Problem with default partition pruning

2019-08-09 Thread Alvaro Herrera
On 2019-Aug-09, Amit Langote wrote: > Hmm, oops. I think that judgement was a bit too rushed on my part. I > unintentionally ended up making the partition constraint to *always* > be fetched, whereas we don't need it in most cases. I've reverted > that change. Yeah, I was quite confused about

Re: SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-09 Thread Alexander Korotkov
On Thu, Aug 8, 2019 at 11:30 PM Alexander Korotkov wrote: > On Thu, Aug 8, 2019 at 11:53 AM Markus Winand wrote: > > The patch makes my tests pass. > > Cool. > > > I wonder about a few things: > > > > - Isn’t there any code that could be re-used for that (the one triggered by > > ‘a’ < ‘A’ COLLA

Re: Rethinking opclass member checks and dependency strength

2019-08-09 Thread Komяpa
> > But none of our contrib modules do it like that, and I'd lay long odds > against any third party code doing it either. Thoughts? > PostGIS has some rarely used box operations as part of GiST opclass, like "overabove". These are source of misunderstanding, as it hinges on the fact that non-

Re: Global temporary tables

2019-08-09 Thread Konstantin Knizhnik
On 09.08.2019 8:34, Craig Ringer wrote: On Thu, 8 Aug 2019 at 15:03, Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote: On 08.08.2019 5:40, Craig Ringer wrote: On Tue, 6 Aug 2019 at 16:32, Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote: New ver

Re: Store FullTransactionId in TwoPhaseFileHeader/GlobalTransactionData

2019-08-09 Thread Robert Haas
On Wed, Aug 7, 2019 at 6:56 AM vignesh C wrote: > Going by the discussion shall we conclude that we don't need to > convert the subxids into fxid's as part of this fix. > Let me know if any further changes need to be done. I'm not sure, but I think the prior question is whether we want this patch

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Thu, Aug 08, 2019 at 11:16:24PM -0700, Jeff Davis wrote: > > On Fri, 2019-08-09 at 12:00 +0900, Michael Paquier wrote: > > > What about auth_protocol then? It seems to me that it could be > > > useful > > > to have the restriction on

Re: block-level incremental backup

2019-08-09 Thread Robert Haas
On Thu, Aug 8, 2019 at 8:37 PM Jeevan Ladhe wrote: > + if (!XLogRecPtrIsInvalid(previous_lsn)) > + appendStringInfo(labelfile, "PREVIOUS WAL LOCATION: %X/%X\n", > +(uint32) (previous_lsn >> 32), (uint32) > previous_lsn); > > May be we should rename to s

Re: block-level incremental backup

2019-08-09 Thread Robert Haas
On Wed, Aug 7, 2019 at 5:46 AM Jeevan Chalke wrote: > So, do you mean we should just do fread() and fwrite() for the whole file? > > I thought it is better if it was done by the OS itself instead of reading 1GB > into the memory and writing the same to the file. Well, 'cp' is just a C program. I

Re: SegFault on 9.6.14

2019-08-09 Thread Robert Haas
On Wed, Aug 7, 2019 at 5:45 AM vignesh C wrote: > I have made a patch based on the above lines. > I have tested the scenarios which Thomas had shared in the earlier > mail and few more tests based on Thomas's tests. > I'm not sure if we will be going ahead with this solution or not. > Let me know

Re: Add "password_protocol" connection parameter to libpq

2019-08-09 Thread Michael Paquier
On Thu, Aug 08, 2019 at 11:16:24PM -0700, Jeff Davis wrote: > On Fri, 2019-08-09 at 12:00 +0900, Michael Paquier wrote: > > What about auth_protocol then? It seems to me that it could be > > useful > > to have the restriction on AUTH_REQ_MD5 as well. > > auth_protocol does sound like a good name.

Re: POC: Cleaning up orphaned files using undo logs

2019-08-09 Thread Amit Kapila
On Fri, Jul 26, 2019 at 9:57 PM Amit Khandekar wrote: > > On Fri, 26 Jul 2019 at 12:25, Amit Kapila wrote: > > I agree with all your other comments. > > Thanks for addressing the comments. Below is the continuation of my > comments from 0014-Allow-execution-and-discard-of-undo-by-background-wo.pa

Re: POC: Cleaning up orphaned files using undo logs

2019-08-09 Thread Amit Kapila
On Tue, Jul 23, 2019 at 8:12 PM Amit Khandekar wrote: > > > > Some further review comments for undoworker.c : > > > +/* Sets the worker's lingering status. */ > +static void > +UndoWorkerIsLingering(bool sleep) > The function name sounds like "is the worker lingering ?". Can w

default_table_access_method is not in sample config file

2019-08-09 Thread Heikki Linnakangas
On 11/04/2019 19:49, Andres Freund wrote: On 2019-04-11 14:52:40 +0300, Heikki Linnakangas wrote: diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index f7f726b5aec..bbcab9ce31a 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -3638,7 +3638,

Re: POC: Cleaning up orphaned files using undo logs

2019-08-09 Thread Amit Kapila
On Mon, Jul 22, 2019 at 8:39 PM Amit Khandekar wrote: > > On Mon, 22 Jul 2019 at 14:21, Amit Kapila wrote: > > I have started review of > 0014-Allow-execution-and-discard-of-undo-by-background-wo.patch. Below > are some quick comments to start with: > > +++ b/src/backend/access/undo/undoworker.c

Re: Small const correctness patch

2019-08-09 Thread Kyotaro Horiguchi
At Thu, 8 Aug 2019 22:56:02 +0300, Mark G wrote in > On Thu, Aug 8, 2019 at 8:51 PM Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > > > > How did you find this? Any special compiler settings? > > > > 16 hours stuck in a plane on an international flight. I was just eyeballing

Re: Small patch to fix build on Windows

2019-08-09 Thread Dmitry Igrishin
пт, 9 авг. 2019 г. в 10:23, Kyotaro Horiguchi : > > At Fri, 9 Aug 2019 09:56:27 +0300, Dmitry Igrishin wrote > in > > пт, 9 авг. 2019 г. в 05:45, Michael Paquier : > > > > > > On Thu, Aug 08, 2019 at 10:46:07PM +0300, Dmitry Igrishin wrote: > > > > This looks nice for a Perl hacker :-). As for m

Re: Problem with default partition pruning

2019-08-09 Thread Amit Langote
Thanks for the comments. On Fri, Aug 9, 2019 at 2:44 PM Kyotaro Horiguchi wrote: > +++ b/src/backend/optimizer/util/plancat.c > @@ -1267,10 +1267,14 @@ get_relation_constraints(PlannerInfo *root, > */ >if (include_partition && relation->rd_rel->relispartition) >{ > ... > +else >

Re: Small patch to fix build on Windows

2019-08-09 Thread Kyotaro Horiguchi
At Fri, 9 Aug 2019 09:56:27 +0300, Dmitry Igrishin wrote in > пт, 9 авг. 2019 г. в 05:45, Michael Paquier : > > > > On Thu, Aug 08, 2019 at 10:46:07PM +0300, Dmitry Igrishin wrote: > > > This looks nice for a Perl hacker :-). As for me, it looks unusual and > > > a bit confusing. I never > > > p