Re: Recording foreign key relationships for the system catalogs

2021-02-01 Thread Joel Jacobson
On Tue, Feb 2, 2021, at 04:27, Tom Lane wrote: >Attachments: >add-catalog-foreign-key-info-2.patch Very nice. I could only find one minor error, found by running the regression-tests, and then using the query below to compare "is_opt" with my own "zero_values" in my tool that derives its value fr

RE: [POC] Fast COPY FROM command for the table with foreign partitions

2021-02-01 Thread tsunakawa.ta...@fujitsu.com
Hello, Andrey-san, From: Tang, Haiying > > Sometimes before i suggested additional optimization [1] which can > > additionally speed up COPY by 2-4 times. Maybe you can perform the > > benchmark for this solution too? ... > But the patch no longer applies, I tried to apply on e42b3c3bd6(2021/1/2

Re: Commitfest 2021-01 is now closed.

2021-02-01 Thread Peter Geoghegan
On Mon, Feb 1, 2021 at 7:16 AM Julien Rouhaud wrote: > > Le lun. 1 févr. 2021 à 22:53, Laurenz Albe a écrit > : >> >> On Mon, 2021-02-01 at 23:33 +0900, Masahiko Sawada wrote: >> > I've closed this commitfest. If you have feedback or comment on my CFM >> > work, please tell me here or by directl

Re: Single transaction in the tablesync worker?

2021-02-01 Thread Peter Smith
On Mon, Feb 1, 2021 at 11:26 PM Amit Kapila wrote: > > I have updated the patch to display WARNING for each of the tablesync > slots during DropSubscription. As discussed, I have moved the drop > slot related code towards the end in AlterSubscription_refresh. Apart > from this, I have fixed one mo

Re: Single transaction in the tablesync worker?

2021-02-01 Thread Ajin Cherian
Another failure I see in my testing On publisher create a big enough table: publisher: postgres=# CREATE TABLE tab_rep (a int primary key);CREATE TABLE postgres=# INSERT INTO tab_rep SELECT generate_series(1,100); INSERT 0 100 postgres=# CREATE PUBLICATION tap_pub FOR ALL TABLES; CREATE PU

Re: row filtering for logical replication

2021-02-01 Thread japin
On Tue, 02 Feb 2021 at 03:11, Euler Taveira wrote: > On Mon, Feb 1, 2021, at 6:11 AM, japin wrote: >> Thanks for updating the patch. Here are some comments: > Thanks for your review. I updated the documentation accordingly. > >> The documentation says: >> >> > Columns used in the WHERE clause

Re: fix typo in reorderbuffer.c

2021-02-01 Thread Michael Paquier
On Wed, Jan 27, 2021 at 07:15:41AM +, Hou, Zhijie wrote: > I agree that it’s better to have a common way, but some different > style of combocid follow the variable or functionname[1]. > We may need to change the var name or function name too. > > [1]: > void > SerializeComboCIDState(Size maxs

Re: Typo in tablesync comment

2021-02-01 Thread Michael Paquier
On Tue, Feb 02, 2021 at 10:38:31AM +1100, Peter Smith wrote: > PSA a trivial patch to correct what seems like a typo in the tablesync > comment. - * subscribed tables and their state. Some transient state during data - * synchronization is kept in shared memory. The states SYNCWAIT

Re: Single transaction in the tablesync worker?

2021-02-01 Thread Ajin Cherian
On Mon, Feb 1, 2021 at 11:26 PM Amit Kapila wrote: > I have updated the patch to display WARNING for each of the tablesync > slots during DropSubscription. As discussed, I have moved the drop > slot related code towards the end in AlterSubscription_refresh. Apart > from this, I have fixed one more

Re: Single transaction in the tablesync worker?

2021-02-01 Thread Amit Kapila
On Tue, Feb 2, 2021 at 8:29 AM Peter Smith wrote: > > On Mon, Feb 1, 2021 at 11:26 PM Amit Kapila wrote: > > > I have updated the patch to display WARNING for each of the tablesync > > slots during DropSubscription. As discussed, I have moved the drop > > slot related code towards the end in Alte

Re: row filtering for logical replication

2021-02-01 Thread Michael Paquier
On Mon, Feb 01, 2021 at 04:11:50PM -0300, Euler Taveira wrote: > After the commit 3696a600e2, the last patch does not apply cleanly. I'm > attaching another version to address the documentation issues. I have bumped into this thread, and applied 0001. My guess is that one of the patches developpe

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 10:31 PM Alvaro Herrera wrote: > > On 2021-Jan-24, Julien Rouhaud wrote: > > > While working on pg14 compatibility for an extension relying on an > > apparently > > uncommon combination of FOR UPDATE and stored function calls, I hit some new > > Asserts introduced in 866e24

Re: New IndexAM API controlling index vacuum strategies

2021-02-01 Thread Peter Geoghegan
On Mon, Feb 1, 2021 at 7:17 PM Peter Geoghegan wrote: > Here is how the triggering criteria could work: maybe skipping > accessing all indexes during VACUUM happens when less than 1% or > 10,000 of the items from the table are to be removed by VACUUM -- > whichever is greater. Of course this is ju

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Julien Rouhaud
On Mon, Feb 01, 2021 at 02:00:48PM -0300, Alvaro Herrera wrote: > On 2021-Jan-24, Julien Rouhaud wrote: > > > While working on pg14 compatibility for an extension relying on an > > apparently > > uncommon combination of FOR UPDATE and stored function calls, I hit some new > > Asserts introduced i

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-01 Thread Fujii Masao
On 2021/02/01 16:39, Bharath Rupireddy wrote: On Mon, Feb 1, 2021 at 12:43 PM Fujii Masao wrote: On 2021/01/27 10:06, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 8:38 AM Bharath Rupireddy wrote: I will post "keep_connections" GUC and "keep_connection" server level option patches late

Re: New IndexAM API controlling index vacuum strategies

2021-02-01 Thread Peter Geoghegan
On Fri, Jan 29, 2021 at 5:26 PM Peter Geoghegan wrote: > It'll be essential to have good instrumentation as we do more > benchmarking. We're probably going to have to make subjective > assessments of benchmark results, based on multiple factors. That will > probably be the only practical way to as

Re: Single transaction in the tablesync worker?

2021-02-01 Thread Peter Smith
On Mon, Feb 1, 2021 at 11:26 PM Amit Kapila wrote: > I have updated the patch to display WARNING for each of the tablesync > slots during DropSubscription. As discussed, I have moved the drop > slot related code towards the end in AlterSubscription_refresh. Apart > from this, I have fixed one mor

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-01 Thread Fujii Masao
On 2021/02/01 16:13, Bharath Rupireddy wrote: On Mon, Feb 1, 2021 at 12:29 PM Fujii Masao wrote: On 2021/01/30 9:28, Bharath Rupireddy wrote: On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: + /* +* It doesn't make sense to show this entry

Re: [PATCH] Doc: improve documentation of oid columns that can be zero. (correct version)

2021-02-01 Thread Euler Taveira
On Sun, Jan 31, 2021, at 10:27 AM, Joel Jacobson wrote: > Here comes the patch again, now including data. Joel, register this patch into the next CF [1] so we don't lose track of it. [1] https://commitfest.postgresql.org/32/ -- Euler Taveira EDB https://www.enterprisedb.com/

RE: Determine parallel-safety of partition relations for Inserts

2021-02-01 Thread Hou, Zhijie
> > IMO, It seems more readable to extract all the check that we can do > > before the safety-check and put them in the new function. > > > > Please consider it for further review. > > > > I've updated your v2 patches and altered some comments and documentation > changes (but made no code changes)

Re: a verbose option for autovacuum

2021-02-01 Thread Euler Taveira
On Fri, Jan 29, 2021, at 4:35 AM, Masahiko Sawada wrote: > I agree to have autovacuum log more information, especially index > vacuums. Currently, the log related to index vacuum is only the number > of index scans. I think it would be helpful if the log has more > details about each index vacuum.

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-01 Thread Michael Paquier
On Mon, Feb 01, 2021 at 06:28:57PM +0300, Alexey Kondratov wrote: > Hm, IIUC, REINDEX CONCURRENTLY doesn't use either of them. It directly uses > index_create() with a proper tablespaceOid instead of > SetRelationTableSpace(). And its checks structure is more restrictive even > without tablespace c

Re: Support for NSS as a libpq TLS backend

2021-02-01 Thread Michael Paquier
On Tue, Feb 02, 2021 at 12:42:23AM +, Jacob Champion wrote: > (Looks like thread safety for NSS_Init was added in 3.13, so we have an > absolute version floor.) If that's the case, I would recommend to add at least something in the section called install-requirements in the docs. > If pgcrypt

Re: a verbose option for autovacuum

2021-02-01 Thread Tommy Li
Hi Masahiko > If we set > it per-table basis, it’s useful when the user already knows which > tables are likely to take a long time for autovacuum I would assume that's the default case, most apps I've seen are designed around a small number of large tables that take up most of the maintenance ef

Re: Support for NSS as a libpq TLS backend

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 21:49 +0100, Daniel Gustafsson wrote: > > Embedded NULLs are now handled in a similar manner to the OpenSSL side, > > though because this failure happens during the certificate > > authentication callback, it results in a TLS alert rather than simply > > closing the connection

Re: Key management with tests

2021-02-01 Thread Bruce Momjian
On Mon, Feb 1, 2021 at 06:31:32PM -0500, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > The purpose of cluster file encryption is to prevent users with read > > access to the directories used to store database files and write-ahead > > log files from being able to acces

RE: Commitfest 2021-01 is now closed.

2021-02-01 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > about 1 month. But if I confirm that the author has a plan to update > the patch soon I didn't close them. So I might have left too many > patches for the next commitfest. If you have a patch that was moved, > and you intend to rewrite enough of it to warrant a resubmission

Re: Support for NSS as a libpq TLS backend

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 21:49 +0100, Daniel Gustafsson wrote: > > On 29 Jan 2021, at 19:46, Jacob Champion wrote: > > I think the bad news is that the static approach will need support for > > ENABLE_THREAD_SAFETY. > > I did some more reading today and noticed that the NSS documentation (and > the

Re: Key management with tests

2021-02-01 Thread Bruce Momjian
On Mon, Feb 1, 2021 at 06:34:53PM -0500, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Sat, Jan 30, 2021 at 08:23:11AM -0500, Tom Kincaid wrote: > > > I propose that we meet to discuss what approach we want to use to move TDE > > > forward.  We then start

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 18:40 -0500, Stephen Frost wrote: > * Jacob Champion (pchamp...@vmware.com) wrote: > > My goal is to get this one single point of reference, for all of the > > auth backends. The LDAP mapping conversation is separate. > > Presumably this would be the DN for SSL then..? Not j

Re: Determine parallel-safety of partition relations for Inserts

2021-02-01 Thread Zhihong Yu
Hi, For v3_0003-reloption-parallel_dml-src.patch : +* Check if parallel_dml_enabled is enabled for the target table, +* if not, skip the safety checks and return PARALLEL_UNSAFE. Looks like the return value is true / false. So the above comment should be adjusted. + if (!RelationGetPar

Re: Determine parallel-safety of partition relations for Inserts

2021-02-01 Thread Greg Nancarrow
On Mon, Feb 1, 2021 at 4:02 PM Hou, Zhijie wrote: > > Attatching v2 patch which addressed the comments above. > > Some further refactor: > > Introducing a new function is_parallel_possible_for_modify() which decide > whether to do safety check. > > IMO, It seems more readable to extract all the c

Re: Should we make Bitmapsets a kind of Node?

2021-02-01 Thread Tom Lane
David Rowley writes: > On Tue, 2 Feb 2021 at 09:23, Tom Lane wrote: >> Now that commit f003a7522 did away with the partitioned_rels fields, >> my original motivation for doing $SUBJECT is gone. It might still be >> worth doing, but I'm not planning to tackle it right now. > I'm not sure if the

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Stephen Frost
Greetings, * Jacob Champion (pchamp...@vmware.com) wrote: > On Mon, 2021-02-01 at 18:01 -0500, Stephen Frost wrote: > > Ok.. but what's 'go' mean here? We already have views and such for GSS > > and SSL, is the idea to add another view for LDAP and add in columns > > that are returned by pg_stat

Typo in tablesync comment

2021-02-01 Thread Peter Smith
PSA a trivial patch to correct what seems like a typo in the tablesync comment. Kind Regards, Peter Smith. Fujitsu Australia fix-tablesync-comment-typo-20210202.patch Description: Binary data

Re: Key management with tests

2021-02-01 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Sat, Jan 30, 2021 at 08:23:11AM -0500, Tom Kincaid wrote: > > I propose that we meet to discuss what approach we want to use to move TDE > > forward.  We then start a new thread with a proposal on the approach > > and finalize it via commun

Re: Key management with tests

2021-02-01 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Fri, Jan 29, 2021 at 05:40:37PM -0500, Stephen Frost wrote: > > I hope it's pretty clear that I'm also very much in support of both this > > effort with the KMS and of TDE in general- TDE is specifically, > > Yes, thanks. I know we have p

Re: Fix DROP TABLESPACE on Windows with ProcSignalBarrier?

2021-02-01 Thread Thomas Munro
On Tue, Feb 2, 2021 at 11:16 AM Thomas Munro wrote: > ... A straw-man idea > would be to touch a file under PGDATA/pg_dropped and fsync it so it > survives a power outage, have checkpoints clean that out, and have > GetNewRelFileNode() to try access() it. ... I should add, the reason I mentione

Re: memory leak in auto_explain

2021-02-01 Thread Jeff Janes
On Mon, Feb 1, 2021 at 6:09 PM Jeff Janes wrote: > > > create or replace function gibberish(int) returns text language SQL as $_$ > select left(string_agg(md5(random()::text),),$1) from > generate_series(0,$1/32) $_$; > > create table j1 as select x, md5(random()::text) as t11, gibberish(1500

memory leak in auto_explain

2021-02-01 Thread Jeff Janes
I accidentally tried to populate a test case while auto_explain.log_min_duration was set to zero. auto_explain.log_nested_statements was also on. create or replace function gibberish(int) returns text language SQL as $_$ select left(string_agg(md5(random()::text),),$1) from generate_series(0,

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 18:01 -0500, Stephen Frost wrote: > Ok.. but what's 'go' mean here? We already have views and such for GSS > and SSL, is the idea to add another view for LDAP and add in columns > that are returned by pg_stat_get_activity() which are then pulled out by > that view? Or did y

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Stephen Frost
Greetings, * Jacob Champion (pchamp...@vmware.com) wrote: > On Mon, 2021-02-01 at 17:01 -0500, Stephen Frost wrote: > > * Jacob Champion (pchamp...@vmware.com) wrote: > > > And I'm not holding > > > my breath for LDAP servers to start implementing federated identity, > > > though that would be nic

Re: Should we make Bitmapsets a kind of Node?

2021-02-01 Thread David Rowley
On Tue, 2 Feb 2021 at 09:23, Tom Lane wrote: > > Now that commit f003a7522 did away with the partitioned_rels fields, > my original motivation for doing $SUBJECT is gone. It might still be > worth doing, but I'm not planning to tackle it right now. I'm not sure if the misuse of Lists to store no

Re: POC: Cleaning up orphaned files using undo logs

2021-02-01 Thread Bruce Momjian
On Fri, Jan 29, 2021 at 06:30:15PM +0100, Antonin Houska wrote: > Antonin Houska wrote: > > Well, on repeated run of the test I could also hit the first one. I could > > fix > > it and will post a new version of the patch (along with some other small > > changes) this week. > > Attached is the n

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-02-01 Thread Peter Geoghegan
On Mon, Feb 1, 2021 at 1:19 PM Michail Nikolaev wrote: > It is fine to receive a page to the standby from any source: `btpo_flags` > should have some kind “LP_DEAD safe for standby” bit set to allow new bits to > be set and old - read. > > > We can't really mask LP_DEAD bits from > > the primary

Re: Key management with tests

2021-02-01 Thread Bruce Momjian
On Sat, Jan 30, 2021 at 08:23:11AM -0500, Tom Kincaid wrote: > I propose that we meet to discuss what approach we want to use to move TDE > forward.  We then start a new thread with a proposal on the approach > and finalize it via community consensus. I will invite Bruce, Stephen and > Masahiko to

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 17:01 -0500, Stephen Frost wrote: > * Jacob Champion (pchamp...@vmware.com) wrote: > > And I'm not holding > > my breath for LDAP servers to start implementing federated identity, > > though that would be nice. > > Not sure exactly what you're referring to here but AD already

Announcing Release 12 of the PostgreSQL Buildfarm client

2021-02-01 Thread Andrew Dunstan
At long last I have just pushed Release 12 of the PostgreSQL Buildfarm client. It's been about 16 months since the last release. Apart from some minor fixes and code tidy up. this includes the following more notable changes: * the TextUpgradeXVersion module is brought up to date with various

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 18:44 +0100, Magnus Hagander wrote: > What people would *really* want I think is "alow auto-creation of new > roles, and then look up which other roles they should be members of > using ldap" (or "using this script over here" for a more flexible > approach). Which is of course

Re: Fix DROP TABLESPACE on Windows with ProcSignalBarrier?

2021-02-01 Thread Thomas Munro
On Tue, Feb 2, 2021 at 8:02 AM Andres Freund wrote: > It's probably rare enough to care, but this still made me thing whether > we could avoid the checkpoint at all somehow. Requiring an immediate > checkpoint for dropping relations is quite a heavy hammer that > practically cannot be used in prod

Re: Key management with tests

2021-02-01 Thread Bruce Momjian
On Fri, Jan 29, 2021 at 05:05:06PM +0900, Masahiko Sawada wrote: > TBH I’m confused a bit about the recent situation of this patch, but > I Yes, it is easy to get confused. > can contribute to KMS work by discussing, writing, reviewing, and > testing the patch. Also, I can work on the data encryp

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Magnus Hagander
On Mon, Feb 1, 2021 at 10:36 PM Jacob Champion wrote: > > On Sun, 2021-01-31 at 12:27 +0100, Magnus Hagander wrote: > > > (There's also the fact that I think pg_ident mapping for LDAP would be > > > just as useful as it is for GSS or certs. That's for a different > > > conversation.) > > > > Speci

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Stephen Frost
Greetings, * Jacob Champion (pchamp...@vmware.com) wrote: > On Mon, 2021-02-01 at 11:49 -0500, Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > > > But yes, I think the enforced cleartext password proxying is at the > > > core of the problem. LDAP also encourages the idea

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Jacob Champion
On Mon, 2021-02-01 at 11:49 -0500, Stephen Frost wrote: > * Magnus Hagander (mag...@hagander.net) wrote: > > But yes, I think the enforced cleartext password proxying is at the > > core of the problem. LDAP also encourages the idea of centralized > > password-reuse, which is not exactly a great thi

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Jacob Champion
On Sun, 2021-01-31 at 12:27 +0100, Magnus Hagander wrote: > > (There's also the fact that I think pg_ident mapping for LDAP would be > > just as useful as it is for GSS or certs. That's for a different > > conversation.) > > Specifically for search+bind, I would assume? Even for the simple bind c

Re: [sqlsmith] Failed assertion during partition pruning

2021-02-01 Thread David Rowley
On Tue, 2 Feb 2021 at 08:58, Tom Lane wrote: > I renamed things that way, did some more work on the comments, > and pushed it. Thanks for reviewing! Thanks for working on this and coming up with the idea to nuke partitioned_rels. David

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-02-01 Thread Michail Nikolaev
Hello, Peter. Thanks a lot for your comments. There are some mine thoughts, related to the “masked bits” solution and your comments: > During recovery, we will probably always have to consider the > possibility that LP_DEAD bits that get set on the primary may be > received by a replica through s

Re: [HACKERS] Custom compression methods

2021-02-01 Thread Robert Haas
Some more review comments: 'git am' barfs on v0001 because it's got a whitespace error. VARFLAGS_4B_C() doesn't seem to be used in any of the patches. I'm OK with keeping it even if it's not used just because maybe someone will need it later but, uh, don't we need to use it someplace? To avoid m

Re: [PATCH] pg_hba.conf error messages for logical replication connections

2021-02-01 Thread Paul Martinez
On Fri, Jan 29, 2021 at 8:41 PM Amit Kapila wrote: > > Yeah, hints or more details might improve the situation but I am not > sure we want to add more branching here. Can we write something > similar to HOSTNAME_LOOKUP_DETAIL for hints? Also, I think what you > are proposing to write is more of a

Re: Support for NSS as a libpq TLS backend

2021-02-01 Thread Daniel Gustafsson
> On 1 Feb 2021, at 14:25, Michael Paquier wrote: > I have refreshed the docs on top to be consistent with the new > configuration, and applied it after more checks. Thanks, I was just about to send a rebased version earlier today with the doc changes in the 0001 patch when this email landed in

Re: Support for NSS as a libpq TLS backend

2021-02-01 Thread Daniel Gustafsson
> On 29 Jan 2021, at 19:46, Jacob Champion wrote: > I think the bad news is that the static approach will need support for > ENABLE_THREAD_SAFETY. I did some more reading today and noticed that the NSS documentation (and their sample code for doing crypto without TLS connections) says to use NSS

Re: Should we make Bitmapsets a kind of Node?

2021-02-01 Thread Tom Lane
Now that commit f003a7522 did away with the partitioned_rels fields, my original motivation for doing $SUBJECT is gone. It might still be worth doing, but I'm not planning to tackle it right now. regards, tom lane

Re: Recording foreign key relationships for the system catalogs

2021-02-01 Thread Tom Lane
"Joel Jacobson" writes: > The is_array OUT parameter doesn't say which of the possibly many fkcols that > is the array column. Yeah, I didn't write the sgml docs yet, but the comments explain that the array is always the last fkcol. Maybe someday that won't be general enough, but we can cross t

Re: [sqlsmith] Failed assertion during partition pruning

2021-02-01 Thread Tom Lane
David Rowley writes: > On Mon, 1 Feb 2021 at 18:57, Tom Lane wrote: >> Oh, it's too late at night. I now remember that the real problem >> I had with that representation was that it cannot work for joinrels. >> Currently we only apply this logic to partitioned baserels, but >> don't you think it

Re: Recording foreign key relationships for the system catalogs

2021-02-01 Thread Joel Jacobson
On Mon, Feb 1, 2021, at 20:33, Joel Jacobson wrote: >Suggestions on how to fix: > >* Make is_array an boolean[], and let each element represent the is_array >value for each fkcols element. > >* Change interface to be more like information_schema, and add a >"ordinal_position" column, and return e

Re: Recording foreign key relationships for the system catalogs

2021-02-01 Thread Joel Jacobson
Hi again, After trying to use pg_get_catalog_foreign_keys() to replace what I had before, I notice one ambiguity which I think is a serious problem in the machine-readable context. The is_array OUT parameter doesn't say which of the possibly many fkcols that is the array column. One example:

Re: row filtering for logical replication

2021-02-01 Thread Euler Taveira
On Mon, Feb 1, 2021, at 6:11 AM, japin wrote: > Thanks for updating the patch. Here are some comments: Thanks for your review. I updated the documentation accordingly. > The documentation says: > > > Columns used in the WHERE clause must be part of the > > primary key or be covered by REPLICA

Re: Fix DROP TABLESPACE on Windows with ProcSignalBarrier?

2021-02-01 Thread Andres Freund
Hi, Thanks for developing this. On 2021-01-31 02:11:06 +1300, Thomas Munro wrote: > --- a/src/backend/commands/tablespace.c > +++ b/src/backend/commands/tablespace.c > @@ -520,15 +520,23 @@ DropTableSpace(DropTableSpaceStmt *stmt) >* but we can't tell them apart from important dat

Re: SEARCH and CYCLE clauses

2021-02-01 Thread Pavel Stehule
po 1. 2. 2021 v 19:02 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2020-11-25 20:35, Pavel Stehule wrote: > > I checked this patch, and I didn't find any issue. > > > > make check-world passed > > make doc passed > > > > I'll mark it as ready for committer > > This

Re: SEARCH and CYCLE clauses

2021-02-01 Thread Peter Eisentraut
On 2020-11-25 20:35, Pavel Stehule wrote: I checked this patch, and I didn't find any issue. make check-world passed make doc passed I'll mark it as ready for committer This has been committed. Thanks. -- Peter Eisentraut 2ndQuadrant, an EDB company https://www.2ndquadrant.com/

Re: [POC] verifying UTF-8 using SIMD instructions

2021-02-01 Thread Heikki Linnakangas
On 01/02/2021 19:32, John Naylor wrote: It makes sense to start with the ascii subset of UTF-8 for a couple reasons. First, ascii is very widespread in database content, particularly in bulk loads. Second, ascii can be validated using the simple SSE2 intrinsics that come with (I believe) any x6

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Magnus Hagander
On Mon, Feb 1, 2021 at 6:32 PM Tom Lane wrote: > > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> This doesn't sound particularly workable: how would you manage > >> inside-the-database permissions? Kerberos isn't going to know > >> what "view foo" is, let alone know wheth

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> This doesn't sound particularly workable: how would you manage > >> inside-the-database permissions? Kerberos isn't going to know > >> what "view foo" is, let alone know whet

[POC] verifying UTF-8 using SIMD instructions

2021-02-01 Thread John Naylor
Hi, As of b80e10638e3, there is a new API for validating the encoding of strings, and one of the side effects is that we have a wider choice of algorithms. For UTF-8, it has been demonstrated that SIMD is much faster at decoding [1] and validation [2] than the standard approach we use. It makes s

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> This doesn't sound particularly workable: how would you manage >> inside-the-database permissions? Kerberos isn't going to know >> what "view foo" is, let alone know whether you should be allowed >> to read or write it. So ISTM th

Next Commitfest Manager.

2021-02-01 Thread Ibrar Ahmed
As Commitfest 2021-01 is now closed. I am volunteering to manage next commitfest. -- Ibrar Ahmed

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Greg Stark writes: > > I wonder if there isn't room to handle this the other way around. To > > configure Postgres to not need a CREATE ROLE for every role but > > delegate the user management to the external authentication service. > > > So Po

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Alvaro Herrera
On 2021-Jan-24, Julien Rouhaud wrote: > While working on pg14 compatibility for an extension relying on an apparently > uncommon combination of FOR UPDATE and stored function calls, I hit some new > Asserts introduced in 866e24d47db (Extend amcheck to check heap pages): > > + /* > + * Do

Re: Proposal: Save user's original authenticated identity for logging

2021-02-01 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Sat, Jan 30, 2021 at 12:21 AM Jacob Champion wrote: > > > I'm also just generally not thrilled with > > > putting much effort into LDAP as it's a demonstrably insecure > > > authentication mechanism. > > > > Because Postgres has to pr

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-01 Thread Justin Pryzby
On Mon, Feb 01, 2021 at 06:28:57PM +0300, Alexey Kondratov wrote: > On 2021-01-30 05:23, Michael Paquier wrote: > > This makes me really wonder if we would not be better to restrict this > > operation for partitioned relation as part of REINDEX as a first step. > > Another thing, mentioned upthread

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-01 Thread Alexey Kondratov
On 2021-01-30 05:23, Michael Paquier wrote: On Fri, Jan 29, 2021 at 08:56:47PM +0300, Alexey Kondratov wrote: On 2021-01-28 14:42, Alexey Kondratov wrote: No, you are right, we are doing REINDEX with AccessExclusiveLock as it was before. This part is a more specific one. It only applies to par

Re: Commitfest 2021-01 is now closed.

2021-02-01 Thread Julien Rouhaud
Le lun. 1 févr. 2021 à 22:53, Laurenz Albe a écrit : > On Mon, 2021-02-01 at 23:33 +0900, Masahiko Sawada wrote: > > I've closed this commitfest. If you have feedback or comment on my CFM > > work, please tell me here or by directly emailing me. > > I think you did an excellent job. > definitely

Re: Commitfest 2021-01 is now closed.

2021-02-01 Thread Laurenz Albe
On Mon, 2021-02-01 at 23:33 +0900, Masahiko Sawada wrote: > I've closed this commitfest. If you have feedback or comment on my CFM > work, please tell me here or by directly emailing me. I think you did an excellent job. Yours, Laurenz Albe

Commitfest 2021-01 is now closed.

2021-02-01 Thread Masahiko Sawada
Hi all, Commitfest 2021-01 is now closed. All patches have now been dealt with. Before starting on moving and closing patches the stats were: Needs review: 146 (-4) Waiting on Author: 25 (+1) Ready for Committer : 23 (-1) Committed: 56 (+4) Moved to next CF: 2 (+2) Withdrawn: 8 (+0) The followin

Re: Add primary keys to system catalogs

2021-02-01 Thread Tom Lane
Peter Eisentraut writes: > On 2021-01-30 22:56, Tom Lane wrote: >> Hmm, shouldn't there have been a catversion bump in there? > I suppose yes on the grounds that it introduces something new in a > freshly initdb-ed database. But I thought it wasn't necessary because > there is no dependency be

Re: [BUG] orphaned function

2021-02-01 Thread Masahiko Sawada
On Mon, Feb 1, 2021 at 11:06 PM Drouvot, Bertrand wrote: > > Hi Sawada, > > On 2/1/21 2:37 PM, Masahiko Sawada wrote: > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and know > > the content is safe

Re: Recording foreign key relationships for the system catalogs

2021-02-01 Thread Joel Jacobson
Could it be an idea to also add OUT can_be_zero boolean to pg_get_catalog_foreign_keys()'s out parameters? This information is useful to know if one should be doing an INNER JOIN or a LEFT JOIN on the foreign keys. The information is mostly available in the documentation already, but not qu

Re: avoid bitmapOR-ing indexes with scan condition inconsistent with partition constraint

2021-02-01 Thread Masahiko Sawada
On Mon, Feb 1, 2021 at 12:09 PM Masahiko Sawada wrote: > > Hi, > > On Fri, Nov 13, 2020 at 5:14 AM Tom Lane wrote: > > > > I started looking through this patch. I really quite dislike solving > > this via a kluge in indxpath.c. There are multiple disadvantages > > to that: > > > > * It only hel

Re: [BUG] orphaned function

2021-02-01 Thread Masahiko Sawada
On Mon, Feb 1, 2021 at 11:26 AM Masahiko Sawada wrote: > > Hi Bertrand, > > On Fri, Dec 18, 2020 at 6:52 PM Gilles Darold wrote: > > > > Le 18/12/2020 à 00:26, Tom Lane a écrit : > > > Gilles Darold writes: > > >> The same problem applies if the returned type or the procedural language > > >> is

Re: bitmaps and correlation

2021-02-01 Thread Masahiko Sawada
On Thu, Jan 28, 2021 at 9:51 PM Masahiko Sawada wrote: > > On Sat, Nov 28, 2020 at 5:49 AM Tom Lane wrote: > > > > Heikki Linnakangas writes: > > > Other than that, and a quick pgdindent run, this seems ready to me. I'll > > > mark it as Ready for Committer. > > > > I dunno, this seems largely m

Re: Recording foreign key relationships for the system catalogs

2021-02-01 Thread Joel Jacobson
Very nice. Thanks to this patch, I can get rid of my own parse-catalogs.pl hack and use pg_get_catalog_foreign_keys() instead. +1 I can with high confidence assert the correctness of pg_get_catalog_foreign_keys()'s output, as it matches the lookup tables for the tool I'm hacking on precisely:

Re: [PATCH] remove deprecated v8.2 containment operators

2021-02-01 Thread Masahiko Sawada
On Thu, Jan 28, 2021 at 9:50 PM Masahiko Sawada wrote: > > On Tue, Dec 1, 2020 at 4:09 AM Tom Lane wrote: > > > > Justin Pryzby writes: > > > I think this is waiting on me to provide a patch for the contrib/ modules > > > with > > > update script removing the SQL operators, and documentating th

Re: Support for NSS as a libpq TLS backend

2021-02-01 Thread Michael Paquier
On Fri, Jan 29, 2021 at 01:57:02PM +0100, Daniel Gustafsson wrote: > This has been discussed elsewhere in the thread, so let's continue that there. > The attached v23 does however split off --with-ssl for OpenSSL in 0001, adding > the nss option in 0002. While going through 0001, I have found a co

Re: Fix typo about generate_gather_paths

2021-02-01 Thread Masahiko Sawada
Hi, On Mon, Feb 1, 2021 at 11:44 AM Masahiko Sawada wrote: > > Hi, > > On Wed, Dec 23, 2020 at 3:24 AM Tomas Vondra > wrote: > > > > On 12/9/20 3:21 AM, Hou, Zhijie wrote: > > > Hi > > > > > > Since ba3e76c, > > > the optimizer call generate_useful_gather_paths instead of > > > generate_gather_

Re: pgbench stopped supporting large number of client connections on Windows

2021-02-01 Thread Masahiko Sawada
Hi, On Thu, Jan 7, 2021 at 10:48 PM Masahiko Sawada wrote: > > Hi Marina, > > On Sun, Nov 8, 2020 at 11:59 PM Marina Polyakova > wrote: > > > > On 2020-11-07 01:01, Fabien COELHO wrote: > > > Hello Marina, > > > > Hello, Fabien! > > > > Thank you for your comments! > > > > >> While trying to te

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-02-01 Thread Yugo NAGATA
On Mon, 1 Feb 2021 13:28:45 +0800 Julien Rouhaud wrote: > On Thu, Jan 28, 2021 at 8:38 PM Yugo NAGATA wrote: > > > > postgres=# explain (analyze, verbose) select * from a,b where a.i=b.j; > > > > QUERY PLAN > >

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-01 Thread Greg Nancarrow
On Mon, Feb 1, 2021 at 8:19 PM Hou, Zhijie wrote: > > > When developing the reloption patch, I noticed some issues in the patch. > > 1). > > - Reduce Insert parallel-safety checks required for some SQL, by noting > > that the subquery must operate on a relation (check for RTE_RELATION in > > subqu

Re: Single transaction in the tablesync worker?

2021-02-01 Thread Amit Kapila
On Mon, Feb 1, 2021 at 11:23 AM Peter Smith wrote: > > On Mon, Feb 1, 2021 at 3:44 PM Amit Kapila wrote: > > > > On Mon, Feb 1, 2021 at 9:38 AM Peter Smith wrote: > > > > No, there is a functionality change as well. The way we have code in > > v22 can easily lead to a problem when we have droppe

Re: PATCH: Attempt to make dbsize a bit more consistent

2021-02-01 Thread Masahiko Sawada
On Thu, Nov 12, 2020 at 2:54 AM Soumyadeep Chakraborty wrote: > > Hey Georgios, > > On Tue, Nov 10, 2020 at 6:20 AM wrote: > > > > > > > > > > > > > > ‐‐‐ Original Message ‐‐‐ > > On Monday, November 9, 2020 7:50 PM, Soumyadeep Chakraborty > > wrote: > > > > > Hey Georgios, > > > > > >

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 4:05 PM Julien Rouhaud wrote: > > On Mon, Feb 1, 2021 at 2:05 PM Dilip Kumar wrote: > > > > On Sun, Jan 24, 2021 at 9:31 PM Alvaro Herrera > > wrote: > > > > > > On 2021-Jan-24, Julien Rouhaud wrote: > > > > > > > + /* > > > > + * Do not allow tuples with invalid

  1   2   >