Re: BUG #16079: Question Regarding the BUG #16064

2020-12-21 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Dec 21, 2020 at 8:06 PM Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > > > On Mon, Dec 21, 2020 at 7:44 PM Tom Lane wrote: > > > > BTW, do we have a client-side setting t

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-12-23 Thread Stephen Frost
umns." > > On Mon, Nov 23, 2020 at 06:06:19PM -0500, Tom Lane wrote: > > I'm mostly concerned about removing the isdir output of pg_stat_file(). > > Maybe we could compromise to the extent of keeping that, allowing it > > to be partially duplicative of a file-type-c

Re: [PATCH] Simplify permission checking logic in user.c

2020-12-30 Thread Stephen Frost
Greetings, * Paul Martinez (paul...@google.com) wrote: > This small patch simply modifies the code to replace the if/else-if chain with > separate if statements, and always checks whether the user has the CREATEROLE > privilege. (The have_createrole_privilege function includes another superuser >

Re: [PATCH] Simplify permission checking logic in user.c

2020-12-30 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Tue, Dec 29, 2020 at 02:26:19PM -0600, Paul Martinez wrote: > > The checks for whether the current user can create a user with the > > SUPERUSER, > > REPLICATION, or BYPASSRLS attributes are chained together using if/else-if, > > befo

Re: [PATCH] Simplify permission checking logic in user.c

2020-12-30 Thread Stephen Frost
Greetings, * Paul Martinez (paul...@google.com) wrote: > You've identified exactly the problem we're running into -- we want to > allow customers, who aren't superusers, to create replication roles. This is also where it's probably useful to think about what the impact of that is- after all, sinc

Re: Proposed patch for key managment

2020-12-30 Thread Stephen Frost
Greetings, * Fabien COELHO (coe...@cri.ensmp.fr) wrote: > I think that an API should be carefully thought about, without assumption > about the underlying cryptography (algorithm, key lengths, modes, how keys > are derived and stored, and so on), and its usefulness be demonstrated by > actually be

Re: Proposed patch for key managment

2020-12-31 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Wed, Dec 30, 2020 at 06:49:34PM -0500, Stephen Frost wrote: > > The API to fetch the KEK doesn't care at all about where it's stored or > > how it's derived or anything like that. There's a relatively s

Re: Proposed patch for key management

2020-12-31 Thread Stephen Frost
Greetings, * Fabien COELHO (coe...@cri.ensmp.fr) wrote: > >>The API should NOT make assumptions about the cryptographic design, what > >>depends about what, where things are stored… ISTM that Pg should only care > >>about naming keys, holding them when created/retrieved (but not create > >>them),

Re: Proposed patch for key managment

2020-12-31 Thread Stephen Frost
Greetings, * Fabien COELHO (coe...@cri.ensmp.fr) wrote: > >>The implementations should not have to be in any particular language: Shell, > >>Perl, Python, C should be possible. > > > >I disagree that it makes any sense to pass actual encryption out to a > >shell script. > > Yes, sure. I'm talking

Re: Proposed patch for key management

2020-12-31 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Dec 31, 2020 at 11:11:11AM +0100, Fabien COELHO wrote: > > > I am not sure what else I can add to this discussion. Having something > > > that is completely external might be a nice option, but I don't think it > > > is the common use

Re: crash recovery vs partially written WAL

2020-12-31 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Wed, Dec 30, 2020 at 12:52:46PM -0800, Andres Freund wrote: > > A question from a colleague made me wonder if there are scenarios where > > two subsequent crashes could lead to wrong WAL to be applied. > > > > Imagine the following scenari

Re: Move --data-checksums to common options in initdb --help

2021-01-02 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote: > > I think enough people use data checksums these days that it warrants to > > be moved into the "normal part", like in the attached. > > +1. Let's see first what others th

Re: Safety/validity of resetting permissions by updating system tables

2021-01-05 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Andrew Dunstan writes: > > I think there is probably a good case for some sort of "from scratch" > > option on GRANT. > > Maybe my head's not screwed on straight this morning, but it seems > to me that any such action would typically be revokin

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-01-04 19:11:43 +0100, Michael Banck wrote: > > Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost: > > > * Michael Paquier (mich...@paquier.xyz) wrote: > > > > On Fri, Jan 01, 2021 at 08:34:3

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Wed, Jan 6, 2021 at 12:02:40PM -0500, Stephen Frost wrote: > > > It unfortunately also hurts other workloads. If we moved towards a saner > > > compression algorithm that'd perhaps not be an issue anymor

Re: Enhance traceability of wal_level changes for backup management

2021-01-06 Thread Stephen Frost
Greetings, * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > You said > > The use case I imagined is that the user temporarily > > changes wal_level to 'none' from 'replica' or 'logical' to speed up loading > > and > > changes back to the normal. In this case, the backups take

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-01-06 12:02:40 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > On 2021-01-04 19:11:43 +0100, Michael Banck wrote: > > > > Am Samstag, den 02.01.2021, 10:47 -0500 schr

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-01-06 18:27:48 +0100, Magnus Hagander wrote: > > The other argument is that admins can cheaply and quickly turn off > > checksums if they don't want them. > > > > The same cannot be said for turning them *on* again, that's a very >

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-01-06 13:01:59 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > > imv, enabling page checksums is akin to having fsync enabled by default. > > > > Does it impact perf

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Wed, Jan 6, 2021 at 11:44 AM Stephen Frost wrote: > > Having fsync off won't actually cause corruption unless you have an OS > > crash or don't sync the disks when you reboot the system though- so it's >

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Wed, Jan 6, 2021 at 12:03 PM Stephen Frost wrote: > > Do you really believe it to be wrong? Do we stop performing the correct > > write calls in the correct order to the kernel with fsync being off? If > > the kernel

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-07 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Wed, Jan 6, 2021 at 12:30 PM Stephen Frost wrote: > > As already mentioned, it's also, at least today, far > > simpler to disable checksums than to enable them, which is something > > else to consider when think

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-07 Thread Stephen Frost
Greetings, * Michael Banck (michael.ba...@credativ.de) wrote: > Am Mittwoch, den 06.01.2021, 13:08 -0800 schrieb Peter Geoghegan: > > On Wed, Jan 6, 2021 at 1:04 PM Michael Banck > > wrote: > > > At least data_checksums=on for Azure Managed Postgres, Amazon RDS and > > > Google Cloud SQL Postgre

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-07 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Thu, Jan 7, 2021 at 1:14 PM Stephen Frost wrote: > > Much of this line of discussion seems to be, incorrectly, focused on my > > mere mention of viewing the use of fsync and checksums as mechanism for > > addressing ce

Re: Key management with tests

2021-01-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Jan 7, 2021 at 04:08:49PM -0300, Álvaro Herrera wrote: > > On 2021-Jan-07, Bruce Momjian wrote: > > > > > All the tests pass now. The current src/test directory is 19MB, and > > > adding these tests takes it to 23MB, or a 20% increa

Re: Key management with tests

2021-01-08 Thread Stephen Frost
Greetings Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Jan 1, 2021 at 01:07:50AM -0500, Bruce Momjian wrote: > > On Thu, Dec 31, 2020 at 11:50:47PM -0500, Bruce Momjian wrote: > > > I have completed the key management patch with tests created by Stephen > >

Re: Key management with tests

2021-01-08 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Jan 8, 2021 at 03:33:44PM -0500, Stephen Frost wrote: > > > Anyway, I think we need to figure out how to trim. The first part would > > > be to figure out whether we need 128 _and_ 256-bit tests, and then see

Re: Enhance traceability of wal_level changes for backup management

2021-01-08 Thread Stephen Frost
Greetings, * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > On Thursday, January 7, 2021 2:40 AM Stephen Frost wrote: > > * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > > > You said > > > > The use case I imagined is tha

Re: WIP: WAL prefetch (another approach)

2021-02-13 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-02-12 00:42:04 +0100, Tomas Vondra wrote: > > Yeah, that's a good point. I think it'd make sense to keep track of recent > > FPIs and skip prefetching such blocks. But how exactly should we implement > > that, how many blocks do we n

Re: WIP: WAL prefetch (another approach)

2021-02-14 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 2/13/21 10:39 PM, Stephen Frost wrote: > >* Andres Freund (and...@anarazel.de) wrote: > >>On 2021-02-12 00:42:04 +0100, Tomas Vondra wrote: > >>>Yeah, that's a good point. I think it&#

Re: Is a connection max lifetime useful in a connection pool?

2021-02-21 Thread Stephen Frost
Greetings, * Daniele Varrazzo (daniele.varra...@gmail.com) wrote: > I am designing and implementing a connection pool for psycopg3 [1][2]. > Some of the inspiration is coming from HikariCP [3], a Java connection > pool. > > One of the HikariCP configuration parameters is "maxLifetime", whose > de

Re: SSL SNI

2021-02-26 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@enterprisedb.com) wrote: > A customer asked about including Server Name Indication (SNI) into the SSL > connection from the client, so they can use an SSL-aware proxy to route > connections. There was a thread a few years ago where this was briefly

Re: Interest in GSoC 2021 Projects

2021-02-26 Thread Stephen Frost
* and the > potential mentor is *Andreas Scherbaum* and *Stephen Frost*. > > I would like to learn more about these two projects but failed to contact > the mentors. How can I contact them? Also, I really hope to join the > project. Are there any suggestions on application? Glad to he

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-08 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 2/10/21 11:10 PM, Stephen Frost wrote: > > * Heikki Linnakangas (hlinn...@iki.fi) wrote: > >> On 05/02/2021 23:22, Stephen Frost wrote: > >>> Unless there's anything else on this, I'

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-11 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 3/8/21 8:42 PM, Stephen Frost wrote: > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > >> On 2/10/21 11:10 PM, Stephen Frost wrote: > >>> * Heikki Linnakangas (hlinn...@iki.fi) wro

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-11 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 3/12/21 1:11 AM, Stephen Frost wrote: > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > >> On 3/8/21 8:42 PM, Stephen Frost wrote: > >>> * Tomas Vondra (tomas.von...@enterprisedb.com) wro

Re: Enhance traceability of wal_level changes for backup management

2021-03-15 Thread Stephen Frost
Greetings, * tsunakawa.ta...@fujitsu.com (tsunakawa.ta...@fujitsu.com) wrote: > From: David Steele > > As a backup software author, I don't see this feature as very useful. > > > > The problem is that there are lots of ways for WAL to go missing so > > monitoring the WAL archive for gaps is esse

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-16 Thread Stephen Frost
Greetings, * Stephen Frost (sfr...@snowman.net) wrote: > > >> I think e.g. prefetch_targblock could be moved to the next #ifdef, which > > >> will eliminate the one-line ifdef. > > > > > > Sure, done in the attached. > > > > > > Than

Re: automatic analyze: readahead - add "IO read time" log message

2020-10-27 Thread Stephen Frost
Greetings Jakub, * Jakub Wartak (jakub.war...@tomtom.com) wrote: > > The analyze is doing more-or-less random i/o since it's skipping through > > the table picking out select blocks, not doing regular sequential i/o. > VS > >> Breakpoint 1, heapam_scan_analyze_next_block (scan=0x10c8098, > >> blo

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread Stephen Frost
Greetings, * vignesh C (vignes...@gmail.com) wrote: > Log message for GSS connection is missing once connection > authorization is successful. We have similar log messages for SSL > connections once the connection authorization is successful. This > message will help the user to identify the conne

Re: Internal key management system

2020-10-28 Thread Stephen Frost
Greetings, * Craig Ringer (craig.rin...@enterprisedb.com) wrote: > On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: > > TL;DR: > > * Important to check that key rotation is possible on a replica, i.e. > primary and standby can have different cluster passphrase and KE

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-29 Thread Stephen Frost
Greetings, * vignesh C (vignes...@gmail.com) wrote: > I have made a v2 patch based on the changes you have suggested. The > patch for the same is attached. > From b067cf823750f200102be0a0cad9a26a08e29a92 Mon Sep 17 00:00:00 2001 > From: Vignesh C > Date: Wed, 28 Oct 2020 08:19:06 +0530 > Subject

Re: New default role- 'pg_read_all_data'

2020-10-29 Thread Stephen Frost
Greetings, * Georgios Kokolatos (gkokola...@protonmail.com) wrote: > this patch is in "Ready for committer" state and the Cfbot is happy. Glad that's still the case. :) > Is there any committer that is available for taking a look at it? If there aren't any objections or further comments, I'll t

Re: Autovacuum worker doesn't immediately exit on postmaster death

2020-10-29 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Victor Yegorov writes: > > ср, 28 окт. 2020 г. в 19:44, Alexander Kukushkin : > >> I know, nobody in their mind should do that, but, if the postmaster > >> process is killed with SIGKILL signal, most backend processes > >> correctly notice the f

Re: Autovacuum worker doesn't immediately exit on postmaster death

2020-10-29 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2020-10-29 12:27:53 -0400, Tom Lane wrote: > > Maybe put a check into vacuum_delay_point, and poll the pipe when we're > > about to sleep anyway? > > Perhaps we should just replace the pg_usleep() with a latch wait? I'm not sure why, bu

Re: Autovacuum worker doesn't immediately exit on postmaster death

2020-10-30 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Alvaro Herrera writes: > > On 2020-Oct-29, Stephen Frost wrote: > >> I do think it'd be good to find a way to check every once in a while > >> even when we aren't going to delay though. Not sure what the be

Re: public schema default ACL

2020-11-02 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 2020-10-31 17:35, Noah Misch wrote: > >Overall, that's 3.2 votes for (b)(3)(X) and 0.0 to 1.0 votes for changing > >nothing. That suffices to proceed with (b)(3)(X). However, given the few > >votes and the conspicuous n

Re: Disable WAL logging to speed up data loading

2020-11-02 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Nov 2, 2020 at 4:28 PM Robert Haas wrote: > > > > On Thu, Oct 29, 2020 at 4:00 PM Fujii Masao > > wrote: > > > Yes. What I meant was such a safe guard needs to be implemented. > > > > > > This may mean that if we want to recove

Re: public schema default ACL

2020-11-02 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Mon, Nov 2, 2020 at 5:51 AM Peter Eisentraut > > wrote: > >> I'm not convinced, however, that this would would really move the needle > >> in terms of the general security-uneasiness about the public schema and > >> s

Re: automatic analyze: readahead - add "IO read time" log message

2020-11-03 Thread Stephen Frost
Greetings, * Jakub Wartak (jakub.war...@tomtom.com) wrote: > >Interesting that you weren't seeing any benefit to disabling readahead. > > I've got some free minutes and I have repeated the exercise in more realistic > and strict environment that previous one to conclude that the current > situat

Re: automatic analyze: readahead - add "IO read time" log message

2020-11-04 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Wed, Nov 04, 2020 at 09:07:59AM +, Jakub Wartak wrote: > >I saw up 410MB/s for a few seconds with this patch on NVMe, and that's > >huge ~5.2x improvement which is amazing for a such simple patch. Nice! > >The system and da

Re: automatic analyze: readahead - add "IO read time" log message

2020-11-04 Thread Stephen Frost
for VACUUM in a bit. If you're doing further benchmarking of ANALYZE though, this would probably be the better patch to use. Certainly improved performance here quite a bit with effective_io_concurrency set to 16. Thanks, Stephen From b154e180718b4b57433093719a43ee6310d1da6f Mon Sep 17 00

Re: Disable WAL logging to speed up data loading

2020-11-09 Thread Stephen Frost
Greetings, * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > On Tuesday, Nov 3, 2020 3:02 AM Stephen Frost wrote: > > I'm not sure that wal_level=none is really the right way to address this > > use-case. We already have unlogged tables and that's p

Re: Reduce the time required for a database recovery from archive.

2020-11-09 Thread Stephen Frost
Greetings, * Dmitry Shulga (d.shu...@postgrespro.ru) wrote: > > On 19 Oct 2020, at 23:25, Stephen Frost wrote: > >>>> Implementation of this approach assumes running several background > >>>> processes (bgworkers) > >>>> each of wh

Re: Disable WAL logging to speed up data loading

2020-11-09 Thread Stephen Frost
Greetings, * David G. Johnston (david.g.johns...@gmail.com) wrote: > On Mon, Nov 9, 2020 at 8:18 AM Stephen Frost wrote: > > Presently, my feeling is that we could address this use-case without > > having to introduce a new cluster-wide WAL level, and that's the > >

Re: automatic analyze: readahead - add "IO read time" log message

2020-11-09 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On 11/4/20 5:02 PM, Stephen Frost wrote: > >* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > >>>If you highlight "738754560" in the output it appears to duplicate the > >>>sys

Re: Disable WAL logging to speed up data loading

2020-11-09 Thread Stephen Frost
Greetings, * David G. Johnston (david.g.johns...@gmail.com) wrote: > On Mon, Nov 9, 2020 at 10:36 AM Stephen Frost wrote: > > * David G. Johnston (david.g.johns...@gmail.com) wrote: > > > If the commit doesn't complete all of the newly created pages are junk. > >

Re: Disable WAL logging to speed up data loading

2020-11-10 Thread Stephen Frost
Greetings, * Kyotaro Horiguchi (horikyota@gmail.com) wrote: > At Mon, 9 Nov 2020 10:18:08 -0500, Stephen Frost wrote > in > > Greetings, > > > > * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > > > When I consider the use case is the

Re: Disable WAL logging to speed up data loading

2020-11-10 Thread Stephen Frost
Greetings, * Kyotaro Horiguchi (horikyota@gmail.com) wrote: > For fuel(?) of the discussion, I tried a very-quick PoC for in-place > ALTER TABLE SET LOGGED/UNLOGGED and resulted as attached. After some > trials of several ways, I drifted to the following way after poking > several ways. > > 1

Re: proposal: possibility to read dumped table's name from file

2020-11-10 Thread Stephen Frost
Greetings, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > rebase + minor change - using pg_get_line_buf instead pg_get_line_append I started looking at this and went back through the thread and while I tend to agree that JSON may not be a good choice for this, it's not the only possible alter

Re: Disable WAL logging to speed up data loading

2020-11-11 Thread Stephen Frost
Greetings, * tsunakawa.ta...@fujitsu.com (tsunakawa.ta...@fujitsu.com) wrote: > * ALTER TABLE SET UNLOGGED/LOGGED without data copy > Good: > - Does not require server restart (if this feature can be used in all > wal_level settings). > > Bad: > - The user has to maintain and modify some scripts

Re: In-place persistance change of a relation

2020-11-11 Thread Stephen Frost
Greetings, * Kyotaro Horiguchi (horikyota@gmail.com) wrote: > At Tue, 10 Nov 2020 09:33:12 -0500, Stephen Frost wrote > in > > * Kyotaro Horiguchi (horikyota@gmail.com) wrote: > > > For fuel(?) of the discussion, I tried a very-quick PoC for in-place > &g

Re: Allow matching whole DN from a client certificate

2020-11-11 Thread Stephen Frost
Greetings, * Andrew Dunstan (and...@dunslane.net) wrote: > Currently we only match the Common Name (CN) of a client certificate > when authenticating a user. The attached patch allows matching the > entire Distinguished Name (DN) of the certificate. This is enabled by > the HBA line option "client

Re: Add docs stub for recovery.conf

2020-11-12 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Nov 12, 2020 at 10:21:02AM +0800, Craig Ringer wrote: > > Here's how the rendered docs look: https://imgur.com/a/VyjzEw5 > > > > Think. You're used to recovery.conf. You've recently switched to pg 12. You > > search for "recovery.conf

Re: Add docs stub for recovery.conf

2020-11-12 Thread Stephen Frost
Greetings, * Stephen Frost (sfr...@snowman.net) wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, Nov 12, 2020 at 10:21:02AM +0800, Craig Ringer wrote: > > > Here's how the rendered docs look: https://imgur.com/a/VyjzEw5 > > > > > > Th

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-13 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Nov 12, 2020 at 11:28 PM Tom Lane wrote: > > > The changes to the contrib modules appear to be incomplete in some ways. > > > In cube, hstore, and seg, there are no changes to the extension > > > scripts to remove the operators

Re: WIP: WAL prefetch (another approach)

2020-11-13 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 11/13/20 3:20 AM, Thomas Munro wrote: > > I'm not really sure what to do about achive restore scripts that > > block. That seems to be fundamentally incompatible with what I'm > > doing here. > > IMHO we can't do much about th

Re: proposal: possibility to read dumped table's name from file

2020-11-18 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Wed, Nov 11, 2020 at 06:49:43AM +0100, Pavel Stehule wrote: > > >> Perhaps this feature could co-exist with a full blown configuration for > > >> pg_dump, but even then there's certainly issues with what's proposed- > > >> how would you

Re: WIP: WAL prefetch (another approach)

2020-11-18 Thread Stephen Frost
Greetings, * Thomas Munro (thomas.mu...@gmail.com) wrote: > On Sat, Nov 14, 2020 at 4:13 AM Stephen Frost wrote: > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > > > On 11/13/20 3:20 AM, Thomas Munro wrote: > > > > I'm not really sure what to d

Re: Disable WAL logging to speed up data loading

2020-11-19 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > On Thu, 2020-11-19 at 05:24 +, osumi.takami...@fujitsu.com wrote: > > > > > ereport(WARNING, > > > > > (errmsg("WAL was generated with wal_level=minimal, data may > > > > > be missing"), > > > > > errhint("This hap

Re: Disable WAL logging to speed up data loading

2020-11-20 Thread Stephen Frost
Greetings, * osumi.takami...@fujitsu.com (osumi.takami...@fujitsu.com) wrote: > On Friday, Nov 20, 2020 9:33 AM Tsunakawa, Takayuki > wrote: > > From: Kyotaro Horiguchi > > > At Thu, 19 Nov 2020 11:04:17 -0500, Stephen Frost > > > > * Laurenz Albe (laurenz.a..

Re: Online verification of checksums

2020-11-20 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > On 11/20/20 2:28 AM, Michael Paquier wrote: > >On Mon, Nov 16, 2020 at 11:41:51AM +0100, Magnus Hagander wrote: > >>I was referring to the latest patch on the thread. But as I said, I have > >>not read up on all the different issues raised i

Default role -> Predefined role

2020-11-20 Thread Stephen Frost
Greetings, Attached is a patch to move from 'default role' terminology to 'predefined role' in the documentation. In the code, I figured it made more sense to avoid saying either one and instead opted for just 'ROLE_NAME_OF_ROLE' as the #define, which we were very close to (one removing the 'DEFA

Re: Online verification of checksums

2020-11-23 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Fri, Nov 20, 2020 at 11:08:27AM -0500, Stephen Frost wrote: > > David Steele (da...@pgmasters.net) wrote: > >> Our current plan for pgBackRest: > >> > >> 1) Remove the LSN check as you have do

Re: Online verification of checksums

2020-11-23 Thread Stephen Frost
Greetings, * Anastasia Lubennikova (a.lubennik...@postgrespro.ru) wrote: > On 21.11.2020 04:30, Michael Paquier wrote: > >The only method I can think as being really > >reliable is based on two facts: > >- Do a check only on pd_checksums, as that validates the full contents > >of the page. > >- Wh

Re: Online verification of checksums

2020-11-23 Thread Stephen Frost
Greetings, * Anastasia Lubennikova (a.lubennik...@postgrespro.ru) wrote: > On 23.11.2020 18:35, Stephen Frost wrote: > >* Anastasia Lubennikova (a.lubennik...@postgrespro.ru) wrote: > >>On 21.11.2020 04:30, Michael Paquier wrote: > >>>The only method I can think as

Re: New default role- 'pg_read_all_data'

2020-11-23 Thread Stephen Frost
Greetings, * Anastasia Lubennikova (a.lubennik...@postgrespro.ru) wrote: > On 29.10.2020 17:19, Stephen Frost wrote: > >* Georgios Kokolatos (gkokola...@protonmail.com) wrote: > >>this patch is in "Ready for committer" state and the Cfbot is happy. > >Glad th

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-23 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Justin Pryzby writes: > >> This patch has been waiting for input from a committer on the approach I've > >> taken with the patches since March 10, so I'm planning to set to "Ready" - > >> at > >> least ready for senior review. > > I took a qui

Re: Online verification of checksums

2020-11-23 Thread Stephen Frost
Greetings, On Mon, Nov 23, 2020 at 20:28 Michael Paquier wrote: > On Mon, Nov 23, 2020 at 05:28:52PM -0500, Stephen Frost wrote: > > * Anastasia Lubennikova (a.lubennik...@postgrespro.ru) wrote: > >> Yes and this is a tricky part. Until you have explained it in your > l

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-24 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I took a quick look through this. This is just MHO, of course: > >> > >> * I don't think it's okay to change the existing sig

Re: A few new options for CHECKPOINT

2020-11-25 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/24/20, 4:03 PM, "tsunakawa.ta...@fujitsu.com" > wrote: > > From: Bossart, Nathan > >> The main purpose of this patch is to give users more control over their > >> manually > >> requested checkpoints or restartpoints. I suspect

Re: Online verification of checksums

2020-11-27 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Nov 26, 2020 at 8:42 AM Michael Paquier wrote: > > On Tue, Nov 24, 2020 at 12:38:30PM -0500, David Steele wrote: > > > We are not just looking at one LSN value. Here are the steps we are > > > proposing (I'll skip checks for zero

Re: A few new options for CHECKPOINT

2020-11-27 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > Thanks to all for the feedback. I've attached v2 of the patch. I've > removed the WAIT and FORCE options and renamed IMMEDIATE to FAST. > > On 11/25/20, 7:52 AM, "Stephen Frost" wrote: > > I&#x

Re: proposal: possibility to read dumped table's name from file

2020-11-27 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Dean Rasheed writes: > > Actually, that raises a different possible benefit of passing options > > in an options file -- if the user wants to pass in a table name > > pattern, it can be a nuisance if the shell's argument processing does > > addi

Re: proposal: possibility to read dumped table's name from file

2020-11-27 Thread Stephen Frost
Greetings, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > I agree that being able to configure pg_dump via a config file would > > be very useful, but the syntax proposed here feels much more like a > > hacked-up syntax designed to meet this one use case, rather than a > > good general-purpo

Re: A few new options for CHECKPOINT

2020-11-27 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/27/20, 8:29 AM, "Stephen Frost" wrote: > > Also note that, in all other cases (that is, when there *is* outstanding > > WAL since the last checkpoint), pg_start_backup actually just waits for >

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-27 Thread Stephen Frost
sh it. Thanks, Stephen From f78100d1c7401d4d47e6ee58f1baaac5b21b1216 Mon Sep 17 00:00:00 2001 From: Stephen Frost Date: Fri, 27 Nov 2020 16:52:44 -0500 Subject: [PATCH] Add GSS information to connection authorized log message GSS information (if used) such as if the connection was authorized using GSS or if it

Re: A few new options for CHECKPOINT

2020-11-28 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/27/20, 10:58 AM, "Stephen Frost" wrote: > > If you'd like to show that I'm wrong, and it's entirely possible that I > > am, then retry the above with actual load on the system, and also &

Re: Add docs stub for recovery.conf

2020-11-29 Thread Stephen Frost
Greetings, * Craig Ringer (craig.rin...@enterprisedb.com) wrote: > On Sat, Nov 14, 2020 at 1:42 AM Bruce Momjian wrote: > > Clearly we have need for documenting these renamings somewhere. We were > > going to go with a simple URL redirect and a "tip" for > > default/pre-installed roles, but I lik

Re: A few new options for CHECKPOINT

2020-11-29 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/28/20, 9:50 AM, "Stephen Frost" wrote: > >> Granted, I used a rather aggressive checkpoint_timeout, but I think > >> this demonstrates that waiting for a non-immediate checkpoint to > >> com

Re: Online verification of checksums

2020-11-30 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Fri, Nov 27, 2020 at 11:15:27AM -0500, Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > >> On Thu, Nov 26, 2020 at 8:42 AM Michael Paquier > >> wrote: > >>> But he

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 08:12:00PM +0900, Masahiko Sawada wrote: > > Looking at the patch, it supports three algorithms but only > > PG_CIPHER_AES_KWP is used in the core for now: > > > > +/* > > + * Supported symmetric encryption algorithm.

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 12:54:49PM -0500, Stephen Frost wrote: > > Although, another approach and one that I've discussed a bit with Bruce, > > is to have more keys- such as a key for temporary files, and perhaps >

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 01:23:27PM -0500, Stephen Frost wrote: > > Yes, and it avoids the issue of using a single key for too much, which > > is also a concern. The remaining larger issues are to figure out a > > pla

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 02:19:22PM -0500, Stephen Frost wrote: > > outputs from the GCM encryption and is what provides the integrity / > > authentication of the encrypted data to be able to detect if it's been > >

Re: Key management with tests

2021-01-12 Thread Stephen Frost
Greetings, * Neil Chen (carpenter.nail...@gmail.com) wrote: > On Tue, Jan 12, 2021 at 10:47 AM Stephen Frost wrote: > > This is an interesting question but ultimately I don't think we should > > be looking at this from the perspective of allowing arbitrary changes to >

Re: Key management with tests

2021-01-12 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jan 12, 2021 at 09:40:53PM +0900, Masahiko Sawada wrote: > > > This says: > > > > > > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#Other_requirements > > > > > > wal_log_hints will be enabled auto

Re: Key management with tests

2021-01-12 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jan 12, 2021 at 01:11:29PM -0500, Stephen Frost wrote: > > > I think one big question is that, since we are using a streaming cipher, > > > do we care about hint bit changes showing to users? I actually don&#x

Re: Key management with tests

2021-01-12 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jan 12, 2021 at 01:44:05PM -0500, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > Well, we have eight unused bits in the IV, so we could just increment > > > that for every hint bi

Re: Add docs stub for recovery.conf

2021-01-13 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Dec 4, 2020 at 02:00:23PM -0500, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > Yes, that is pretty much the same thing I was suggesting, except that > > > each rename has its own

<    1   2   3   4   5   6   7   8   9   10   >