Re: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers

2022-01-06 Thread SATYANARAYANA NARLAPURAM
Consider a cluster formation where we have a Primary(P), Sync Replica(S1), and multiple async replicas for disaster recovery and read scaling (within the region and outside the region). In this setup, S1 is the preferred failover target in an event of the primary failure. When a transaction is comm

Re: Make relfile tombstone files conditional on WAL level

2022-01-06 Thread Dilip Kumar
On Thu, Jan 6, 2022 at 1:12 PM Dilip Kumar wrote: > > > > > I think this idea is worth more consideration. It seems like 2^56 > > relfilenodes ought to be enough for anyone, recalling that you can > > only ever have 2^64 bytes of WAL. So if we do this, we can eliminate a > > bunch of code that is

Re: Remove trailing comma from enums

2022-01-06 Thread Kyotaro Horiguchi
At Thu, 6 Jan 2022 12:52:50 +1100, Peter Smith wrote in > On Thu, Jan 6, 2022 at 12:23 PM Tom Lane wrote: > > > > Thomas Munro writes: > > > On Thu, Jan 6, 2022 at 12:56 PM Peter Smith wrote: > > >> These are quite rare in the PG src, so I doubt they are intentional. > > >> PSA a patch to remo

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-01-06 Thread Etsuro Fujita
On Fri, Dec 3, 2021 at 6:07 PM Fujii Masao wrote: > On 2021/11/16 18:55, Etsuro Fujita wrote: > > I changed my mind; I’ll update the patch to ignore the error as > > before, because 1) as far as I know, there are no reports from the > > field concerning that we ignore all kinds of errors in cleani

Re: Column Filtering in Logical Replication

2022-01-06 Thread Amit Kapila
On Mon, Jan 3, 2022 at 8:01 PM Alvaro Herrera wrote: > fetch_remote_table_info() { .. + appendStringInfo(&cmd, + " SELECT pg_catalog.unnest(prattrs)\n" + "FROM pg_catalog.pg_publication p JOIN\n" + " pg_catalog.pg_publication_rel pr ON (p.oid = pr.prpubid)\n" + " WHERE p.pubname IN

Re: Make relfile tombstone files conditional on WAL level

2022-01-06 Thread Thomas Munro
On Thu, Jan 6, 2022 at 9:13 PM Dilip Kumar wrote: > On Thu, Jan 6, 2022 at 1:12 PM Dilip Kumar wrote: > > > I think this idea is worth more consideration. It seems like 2^56 > > > relfilenodes ought to be enough for anyone, recalling that you can > > > only ever have 2^64 bytes of WAL. So if we d

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Julien Rouhaud
On Thu, Jan 06, 2022 at 08:52:23AM +0100, Pavel Stehule wrote: > > I am not sure if there is enough agreement and if there is enough necessity > for this feature. > > In this discussion there were more general questions about future > development of plpgsql (about possible configurations and comp

Re: Column Filtering in Logical Replication

2022-01-06 Thread Amit Kapila
On Mon, Dec 27, 2021 at 10:36 PM Alvaro Herrera wrote: > > Determining that an array has a NULL element seems convoluted. I ended > up with this query, where comparing the result of array_positions() with > an empty array does that. If anybody knows of a simpler way, or any > situations in which

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: > Your opinions are very much welcome! This is a review of the Int64 options patch, "v6-0001-Add-64-bit-GUCs-for-xids.patch" Applies cleanly, with some fuzz, compiles cleanly and passes make check. Patch eyeballs OK, no obvious defects. Tested us

Re: Index-only scan for btree_gist turns bpchar to char

2022-01-06 Thread Japin Li
On Thu, 06 Jan 2022 at 00:34, Tom Lane wrote: > Japin Li writes: >> Here is a patch for POC testing. > > This is certainly not right. You've made gbt_bpchar_consistent > work identically to gbt_text_consistent, but it needs to implement > a test equivalent to bpchareq, ie ignore trailing space

Re: Emit "checkpoint skipped because system is idle" message at LOG level if log_checkpoints is set

2022-01-06 Thread Julien Rouhaud
On Thu, Jan 06, 2022 at 04:34:38PM +0900, Kyotaro Horiguchi wrote: > At Wed, 5 Jan 2022 17:18:06 -0600, Justin Pryzby wrote > in > > > > |commit 6ef2eba3f57f17960b7cd4958e18aa79e357de2f > > |Author: Andres Freund > > |Date: Thu Dec 22 11:31:50 2016 -0800 > > | > > |Skip checkpoints, arch

Re: Logical insert/update/delete WAL records for custom table AMs

2022-01-06 Thread Simon Riggs
On Sun, 31 Oct 2021 at 18:10, Jeff Davis wrote: > I'm looking for some review on the approach and structure before I > polish and test it. Repurposing the logical msg rmgr into a general purpose logical rmgr seems right. Structure looks obvious, which is good. Please pursue this and I will rev

Re: SQL/JSON: functions

2022-01-06 Thread Himanshu Upadhyaya
On Tue, Jan 4, 2022 at 7:32 PM Andrew Dunstan wrote: I have one general question on the below scenario. CREATE TABLE T (Id INTEGER PRIMARY KEY,Jcol CHARACTER VARYING ( 5000 )CHECK ( Jcol IS JSON ) ); insert into T values (1,323); ORACLE is giving an error(check constraint...violated ORA-06512) f

Re: psql: \dl+ to list large objects privileges

2022-01-06 Thread Pavel Luzanov
Justin, Tom, Thanks for the review and the help in splitting the patch into two parts. I wonder if we shouldn't put these new tests in largeobject.sql instead. (That would mean there are two expected-files to change, which is a bit tedious, but it's not very hard.) As suggested, I moved the t

Re: Column Filtering in Logical Replication

2022-01-06 Thread Alvaro Herrera
On 2022-Jan-06, Amit Kapila wrote: > On Mon, Jan 3, 2022 at 8:01 PM Alvaro Herrera wrote: > > > > fetch_remote_table_info() > { > .. > + appendStringInfo(&cmd, > + " SELECT pg_catalog.unnest(prattrs)\n" > + "FROM pg_catalog.pg_publication p JOIN\n" > + " pg_catalog.pg_publication_re

Re: Column Filtering in Logical Replication

2022-01-06 Thread Alvaro Herrera
On 2022-Jan-06, Amit Kapila wrote: > Considering this, don't we need to deal with "For All Tables" and "For > All Tables In Schema .." Publications in this query? The row filter > patch deal with such cases. The row filter patch handles the NULL case > via C code which makes the query relatively s

Re: a misbehavior of partition row movement (?)

2022-01-06 Thread Alvaro Herrera
On 2022-Jan-06, Amit Langote wrote: > On Thu, Jan 6, 2022 at 7:27 AM Alvaro Herrera wrote: > > I have pushed it thinking that we would not backpatch any of this fix. > > However, after running the tests and realizing that I didn't need an > > initdb for either patch, I wonder if maybe the whole

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2022-01-06 Thread Amit Langote
On Thu, Jan 6, 2022 at 3:43 PM Amit Langote wrote: > On Tue, Dec 21, 2021 at 6:20 AM Tom Lane wrote: > > Robert Haas writes: > > > OK ... but my point is that dump and restore does work. So whatever > > > cases pg_get_expr() doesn't work must be cases that aren't needed for > > > that to happen.

Re: row filtering for logical replication

2022-01-06 Thread Euler Taveira
On Thu, Jan 6, 2022, at 1:18 AM, Amit Kapila wrote: > On Thu, Jan 6, 2022 at 8:43 AM Peter Smith wrote: > > > > On Wed, Jan 5, 2022 at 9:52 PM Amit Kapila wrote: > > > > > ... > > > > > Another minor comment: > > > +static bool pgoutput_row_filter(enum ReorderBufferChangeType changetype, > > > >

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Finnerty, Jim
(Maxim) Re: -- If after upgrade page has no free space for special data, tuples are converted to "double xmax" format: xmin became virtual FrozenTransactionId, xmax occupies the whole 64bit. Page converted to new format when vacuum frees enough space. A better way would be to

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 10:05, Julien Rouhaud wrote: > I agree, but on the other hand I don't see how defining a top level block > alias identical for every single plpgsql function would really make sense. > Not every function has a very long name and would benefit from it, and no one > can really

Re: Make relfile tombstone files conditional on WAL level

2022-01-06 Thread Robert Haas
On Thu, Jan 6, 2022 at 3:47 AM Thomas Munro wrote: > Another problem is that relfilenodes are normally allocated with > GetNewOidWithIndex(), and initially match a relation's OID. We'd need > a new allocator, and they won't be able to match the OID in general > (while we have 32 bit OIDs at least

Re: ICU for global collation

2022-01-06 Thread Finnerty, Jim
I didn't notice anything version-specific about the patch. Would any modifications be needed to backport it to pg13 and pg14? After this patch goes in, the big next thing would be to support nondeterministic collations for LIKE, ILIKE and pattern matching operators in general. Is anyone inte

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 14:28 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 10:05, Julien Rouhaud wrote: > > I agree, but on the other hand I don't see how defining a top level block > > alias identical for every single plpgsql function would really make > sense. > > Not every function has a

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
On Thu, 6 Jan 2022 at 13:15, Finnerty, Jim wrote: > > (Maxim) Re: -- If after upgrade page has no free space for special data, > tuples are >converted to "double xmax" format: xmin became virtual >FrozenTransactionId, xmax occupies the whole 64bit. >Page converted to new f

Re: Refactoring of compression options in pg_basebackup

2022-01-06 Thread Robert Haas
On Thu, Jan 6, 2022 at 12:04 AM Michael Paquier wrote: > Yeah. There are cases for both. I just got to wonder whether it > makes sense to allow both server-side and client-side compression to > be used at the same time. That would be a rather strange case, but > well, with the correct set of op

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2022-01-06 Thread Robert Haas
On Mon, Dec 20, 2021 at 4:20 PM Tom Lane wrote: > pg_get_expr doesn't (or didn't) depend on expression_tree_walker, > so there wasn't a problem there before. I am worried that there > might be other code paths, now or in future, that could try to apply > expression_tree_walker/mutator to relpartb

Re: SQL:2011 application time

2022-01-06 Thread Vik Fearing
On 1/5/22 11:03 PM, Corey Huinker wrote: > > There was similar work being done for system periods, which are a bit > simpler but require a side (history) table to be created. This is false. SYSTEM_TIME periods do not need any kind of history. This was one of the problems I had with Surafel's att

Re: ICU for global collation

2022-01-06 Thread Julien Rouhaud
On Thu, Jan 06, 2022 at 01:55:55PM +, Finnerty, Jim wrote: > > I didn't notice anything version-specific about the patch. Would any > modifications be needed to backport it to pg13 and pg14? This is a new feature so it can't be backported. The changes aren't big and mostly touches places th

Re: Support tab completion for upper character inputs in psql

2022-01-06 Thread Tom Lane
Peter Eisentraut writes: > So the ordering of the suggested completions is different. I don't know > offhand how that ordering is determined. Perhaps it's dependent on > locale, readline version, or operating system. In any case, we need to > figure this out to make this test stable. I don'

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2022-01-06 Thread Tom Lane
Amit Langote writes: > Thanks for addressing that in the patch you posted. I guess fixing > only expression_tree_walker/mutator() suffices for now, but curious to > know if it was intentional that you decided not to touch the following > sites: > exprCollation(): it would perhaps make sense to r

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2022-01-06 Thread Jelte Fennema
Attached are 3 patches that address the feedback from Andres about code duplication and splitting up commits. I completely removed internal_cancel now, since it only had one caller at this point. > IMO, this is a new feature not a bug fix IMO this is definitely a bugfix. Nowhere in the libpq

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 15:05, Pavel Stehule wrote: >>čt 6. 1. 2022 v 14:28 odesílatel Joel Jacobson napsal: >>How about using the existing reserved keyword "in" followed by "." (dot) and >>then the function parameter name? >> >>This idea is based on the assumption "in." would always be a syntax

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 16:59 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 15:05, Pavel Stehule wrote: > >>čt 6. 1. 2022 v 14:28 odesílatel Joel Jacobson > napsal: > >>How about using the existing reserved keyword "in" followed by "." (dot) > and then the function parameter name? > >> > >>

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Finnerty, Jim
Re:Most software has a one-stage upgrade model. What you propose would have us install 2 things, with a step in-between, which makes it harder to manage. The intended benefit would be that the code doesn't need to handle the possibility of 2 different XID representations for the indef

Re: SQL/JSON: functions

2022-01-06 Thread Andrew Dunstan
On 1/6/22 06:24, Himanshu Upadhyaya wrote: > I have one general question on the below scenario. > CREATE TABLE T (Id INTEGER PRIMARY KEY,Jcol CHARACTER VARYING ( 5000 > )CHECK ( Jcol IS JSON ) ); > insert into T values (1,323); > ORACLE is giving an error(check constraint...violated ORA-06512) f

Deduplicate min restart_lsn calculation code

2022-01-06 Thread Bharath Rupireddy
Hi, It seems like the two functions ReplicationSlotsComputeRequiredLSN and ReplicationSlotsComputeLogicalRestartLSN more or less does the same thing which makes me optimize (saving 40 LOC) it as attached. I'm pretty much okay if it gets rejected on the grounds that it creates a lot of diff with th

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 17:10, Pavel Stehule wrote: > I understand well, and I don't think it's nice. > > Are there some similar features in other programming languages? It would be similar to "this" in Javascript/Java/C++, but instead using "in" to access function parameters. Currently, we nee

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Tom Lane
Pavel Stehule writes: > čt 6. 1. 2022 v 16:59 odesílatel Joel Jacobson napsal: >> How could the SQL parser have a problem with it, if "in" is currently >> never followed by "." (dot)? > you can check it. It is true, so IN is usually followed by "(", but until > check I am not able to say if ther

Re: terminate called after throwing an instance of 'std::bad_alloc' (llvmjit)

2022-01-06 Thread Justin Pryzby
There's no leak after running for ~5 weeks. $ ps -O lstart,vsize,rss 17930 PID STARTEDVSZ RSS S TTY TIME COMMAND 17930 Tue Nov 30 15:35:26 2021 1019464 117424 S ? 7-04:54:03 postgres: telsasoft ts 192.168.122.13(57640) idle Unless you suggest otherwise , I'

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 17:55, Tom Lane wrote: > Even if it works today, we could be letting ourselves in for future > trouble. The SQL standard is a moving target, and they could easily > standardize some future syntax involving IN that creates a problem here. Perhaps the "in." notation could be

Re: Suggestion: optionally return default value instead of error on failed cast

2022-01-06 Thread Andrew Dunstan
On 1/4/22 22:17, Corey Huinker wrote: > > currently a failed cast throws an error. It would be useful to have a > way to get a default value instead. > > > I've recently encountered situations where this would have been > helpful. Recently I came across some client code: > > CREATE OR

Re: Emit "checkpoint skipped because system is idle" message at LOG level if log_checkpoints is set

2022-01-06 Thread Justin Pryzby
On Wed, Jan 05, 2022 at 10:45:06AM +0530, Dilip Kumar wrote: > +1 to convert to LOG when log_checkpoints is set. On Thu, Jan 06, 2022 at 04:34:38PM +0900, Kyotaro Horiguchi wrote: > Agreed. -1 to just raising elevel of the message. On Thu, Jan 06, 2022 at 06:58:14PM +0800, Julien Rouhaud wrote: >

Re: Suggestion: optionally return default value instead of error on failed cast

2022-01-06 Thread Corey Huinker
On Thu, Jan 6, 2022 at 12:18 PM Andrew Dunstan wrote: > > On 1/4/22 22:17, Corey Huinker wrote: > > > > currently a failed cast throws an error. It would be useful to have a > > way to get a default value instead. > > > > > > I've recently encountered situations where this would have been

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 17:48 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 17:10, Pavel Stehule wrote: > > I understand well, and I don't think it's nice. > > > > Are there some similar features in other programming languages? > > It would be similar to "this" in Javascript/Java/C++, > but

Re: Add jsonlog log_destination for JSON server logs

2022-01-06 Thread Andrew Dunstan
On 1/5/22 02:32, Michael Paquier wrote: > On Sun, Jan 02, 2022 at 01:34:45PM -0800, Andres Freund wrote: >> The tests don't seem to pass on windows: >> https://cirrus-ci.com/task/5412456754315264?logs=test_bin#L47 >> https://api.cirrus-ci.com/v1/artifact/task/5412456754315264/tap/src/bin/pg_ctl/t

Re: SQL:2011 application time

2022-01-06 Thread Paul A Jungwirth
On Thu, Jan 6, 2022 at 6:45 AM Vik Fearing wrote: > > On 1/5/22 11:03 PM, Corey Huinker wrote: > > > > There was similar work being done for system periods, which are a bit > > simpler but require a side (history) table to be created. > > This is false. SYSTEM_TIME periods do not need any kind of

Re: psql: \dl+ to list large objects privileges

2022-01-06 Thread Tom Lane
Pavel Luzanov writes: >> I wonder if we shouldn't put these new tests in largeobject.sql instead. >> (That would mean there are two expected-files to change, which is a bit >> tedious, but it's not very hard.) > I made the same changes to two files in the 'expected' directory: > largeobject.out

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2022-01-06 Thread Peter Geoghegan
On Tue, Dec 21, 2021 at 11:57 PM Masahiko Sawada wrote: > I've looked at the patch and here are comments: Thanks! The patch bitrot again, so attached is a rebased version, v3. > I think we should set message_level. Otherwise, index AM will set an > invalid log level, although any index AM in co

Re: Deduplicate min restart_lsn calculation code

2022-01-06 Thread Alvaro Herrera
On 2022-Jan-06, Bharath Rupireddy wrote: > Hi, > > It seems like the two functions ReplicationSlotsComputeRequiredLSN and > ReplicationSlotsComputeLogicalRestartLSN more or less does the same > thing which makes me optimize (saving 40 LOC) it as attached. I'm > pretty much okay if it gets rejecte

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 18:18 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 17:55, Tom Lane wrote: > > Even if it works today, we could be letting ourselves in for future > > trouble. The SQL standard is a moving target, and they could easily > > standardize some future syntax involving IN

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
> > > > I accept that both issues are valid, and I don't think we can find a good > design that solves both issues, because there are different objective > expectations. > I accept that both issues are valid, and I don't think we can find a **one** good design that solves both issues, because ther

Re: SQL:2011 application time

2022-01-06 Thread Corey Huinker
> > > But > the standard says that dropping system versioning should automatically > drop all historical records (2 under Part 2: Foundation, 11.30 system versioning clause>). That actually makes sense though: when you > do DML we automatically update the start/end columns, but we don't > save co

Storage for multiple variable-length attributes in a single row

2022-01-06 Thread Esteban Zimanyi
Dear all When ingesting mobility (IoT) data into MobilityDB https://mobilitydb.com/ we transform very wide (2K attributes) car mobility data of high frequence (every tenth of a second) from flat format (e.g. CSV) into MobilityDB format in which there is a single record per trip and each of the sig

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 19:03, Pavel Stehule wrote: > The possibility to define a label dynamically is a better solution (not by > some buildin keyword), > because it allows some possibility for the end user to define what he prefers. I'm trying to understand why you think a user-defined notation

Re: Collecting statistics about contents of JSONB columns

2022-01-06 Thread Tomas Vondra
On 1/1/22 16:33, Zhihong Yu wrote: Hi, For patch 1: +   List       *statisticsName = NIL;   /* optional stats estimat. procedure */ I think if the variable is named estimatorName (or something similar), it would be easier for people to grasp its purpose. I agree "statisticsName" might be

Re: Index-only scan for btree_gist turns bpchar to char

2022-01-06 Thread Tom Lane
Japin Li writes: > On Thu, 06 Jan 2022 at 00:34, Tom Lane wrote: >> The minimum-effort fix would be to apply rtrim1 to both strings >> in gbt_bpchar_consistent, but I wonder if we can improve on that >> by pushing the ignore-trailing-spaces behavior further down. >> I didn't look yet at whether g

Re: Collecting statistics about contents of JSONB columns

2022-01-06 Thread Tomas Vondra
On 1/1/22 22:16, Zhihong Yu wrote: Hi, +static JsonPathStats +jsonStatsFindPathStats(JsonStats jsdata, char *path, int pathlen) Stats appears twice in the method name. I think findJsonPathStats() should suffice. It should check `if (jsdata->nullfrac >= 1.0)` as jsonStatsGetPathStatsStr does.

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 20:03 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 19:03, Pavel Stehule wrote: > > The possibility to define a label dynamically is a better solution (not > by some buildin keyword), > > because it allows some possibility for the end user to define what he > prefers.

Re: Collecting statistics about contents of JSONB columns

2022-01-06 Thread Tomas Vondra
On 1/5/22 21:22, Simon Riggs wrote: On Fri, 31 Dec 2021 at 22:07, Tomas Vondra wrote: The patch does something far more elegant - it simply uses stavalues to store an array of JSONB documents, each describing stats for one path extracted from the sampled documents. Sounds good. I'm sure

Re: Bugs in pgoutput.c

2022-01-06 Thread Tom Lane
Amit Kapila writes: > On Thu, Jan 6, 2022 at 3:42 AM Tom Lane wrote: >> Also ... maybe I'm not looking in the right place, but I do not >> see anything anywhere in logical decoding that is taking any lock >> on the relation being processed. How can that be safe? > We don't need to acquire a loc

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 20:24, Pavel Stehule wrote: > But there is nothing similar in standard. > Standard doesn't specify any column or table or label names in the custom > area. I think that's an unfair comparison. This isn't at all the same thing as dictating column or table names. I merely s

Re: Bugs in pgoutput.c

2022-01-06 Thread Robert Haas
On Thu, Jan 6, 2022 at 2:58 PM Tom Lane wrote: > That might be okay for the system catalog entries, but I don't see > how it prevents some other session from dropping the table entirely, > thereby causing the on-disk storage to go away. Is it guaranteed > that logical decoding will never try to f

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 21:04 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 20:24, Pavel Stehule wrote: > > But there is nothing similar in standard. > > Standard doesn't specify any column or table or label names in the > custom area. > > I think that's an unfair comparison. > This isn't at

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-06 Thread Bossart, Nathan
On 12/29/21, 8:44 AM, "Imseih (AWS), Sami" wrote: > In "pg_stat_progress_vacuum", introduce 2 columns: > > * total_index_vacuum : This is the # of indexes that will be vacuumed. Keep > in mind that if failsafe mode kicks in mid-flight to the vacuum, Postgres may > choose to forgo index scans. Th

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Peter Geoghegan
On Tue, Jan 4, 2022 at 9:40 PM Fujii Masao wrote: > Could you tell me what happens if new tuple with XID larger than xid_base + > 0x is inserted into the page? Such new tuple is not allowed to be > inserted into that page? I fear that this patch will have many bugs along these lines. Ex

Re: Make relfile tombstone files conditional on WAL level

2022-01-06 Thread Andres Freund
On 2022-01-06 08:52:01 -0500, Robert Haas wrote: > On Thu, Jan 6, 2022 at 3:47 AM Thomas Munro wrote: > > Another problem is that relfilenodes are normally allocated with > > GetNewOidWithIndex(), and initially match a relation's OID. We'd need > > a new allocator, and they won't be able to match

Re: psql: \dl+ to list large objects privileges

2022-01-06 Thread Pavel Luzanov
On 06.01.2022 21:13, Tom Lane wrote: I made the same changes to two files in the 'expected' directory: largeobject.out and largeobject_1.out. Although I must say that I still can't understand why more than one file with expected result is used for some tests. Because the output sometimes varies

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Robert Haas
On Fri, Dec 17, 2021 at 9:30 PM Peter Geoghegan wrote: > Can we fully get rid of vacuum_freeze_table_age? Maybe even get rid of > vacuum_freeze_min_age, too? Freezing tuples is a maintenance task for > physical blocks, but we use logical units (XIDs). I don't see how we can get rid of these. We k

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 21:38, Pavel Stehule wrote: > I say, semantically - how I understand the meaning of the word "in" is not > good to use it for generic alias of function arguments (when we have out > arguments too). Trying to imagine a situation where you would need a shorthand also for OU

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2022-01-06 Thread Bossart, Nathan
On 12/21/21, 11:42 AM, "Mark Dilger" wrote: > + /* pre-v9.3 lock-only bit pattern */ > + ereport(ERROR, > + (errcode(ERRCODE_DATA_CORRUPTED), > +errmsg_internal("found tuple with HEAP_XMAX_COMMITTED > and" > +

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:56 PM Amit Kapila wrote: > ... > > > 4. > > > +#define IDX_PUBACTION_n 3 > > > + ExprState*exprstate[IDX_PUBACTION_n]; /* ExprState array for row > > > filter. > > > +One per publication action. */ > > > .. > > > .. > > > > > > I think we can have this define out

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:26 PM Amit Kapila wrote: > > On Tue, Jan 4, 2022 at 12:15 PM Peter Smith wrote: > > > > On Fri, Dec 31, 2021 at 12:39 AM houzj.f...@fujitsu.com > > wrote: > > > > 3) v55-0002 > > > > +static bool pgoutput_row_filter_update_check(enum > > > > ReorderBufferChangeType chang

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:34 PM Peter Smith wrote: > > I have reviewed again the source code for v58-0001. > > Below are my review comments. > > Actually, I intend to fix most of these myself for v59*, so this post > is just for records. > > v58-0001 Review Comments > > > 1

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 5:01 PM vignesh C wrote: > ... > 4) Should this be posted as a separate patch in a new thread, as it is > not part of row filtering: > --- a/src/include/parser/parse_node.h > +++ b/src/include/parser/parse_node.h > @@ -79,7 +79,7 @@ typedef enum ParseExprKind > EXPR_

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 1:05 PM wangw.f...@fujitsu.com wrote: > > On Thu, Jan 4, 2022 at 00:54 PM Peter Smith wrote: > > Modified in v58 [1] as suggested > Thanks for updating the patches. > A few comments about v58-0001 and v58-0002. > > v58-0001 > 1. > How about modifying the following loop in c

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 12:54 PM Robert Haas wrote: > On Fri, Dec 17, 2021 at 9:30 PM Peter Geoghegan wrote: > > Can we fully get rid of vacuum_freeze_table_age? Maybe even get rid of > > vacuum_freeze_min_age, too? Freezing tuples is a maintenance task for > > physical blocks, but we use logical

Re: Add jsonlog log_destination for JSON server logs

2022-01-06 Thread Andrew Dunstan
On 1/6/22 13:06, Andrew Dunstan wrote: > On 1/5/22 02:32, Michael Paquier wrote: >> On Sun, Jan 02, 2022 at 01:34:45PM -0800, Andres Freund wrote: >>> The tests don't seem to pass on windows: >>> https://cirrus-ci.com/task/5412456754315264?logs=test_bin#L47 >>> https://api.cirrus-ci.com/v1/artifa

Re: Feature Proposal: Connection Pool Optimization - Change the Connection User

2022-01-06 Thread Todd Hubers
Hi Everyone, I have started working on this: - Benchmarking - increasingly more comprehensive benchmarking - Prototyping - to simulate the change of users (toggling back and forth) - Draft Implementation - of OPTION-1 (New Protocol Message) - (Then: Working with Odyssey and PgBouncer

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 2:45 PM Peter Geoghegan wrote: > But the "freeze early" heuristics work a bit like that anyway. We > won't freeze all the tuples on a whole heap page early if we won't > otherwise set the heap page to all-visible (not all-frozen) in the VM > anyway. I believe that applicati

fix libpq comment

2022-01-06 Thread Euler Taveira
Hi, While checking the PQping code I noticed that pg_ctl does not rely on PQping since commit f13ea95f9e4 (v10) so the attached patch removes a comment from internal_ping(). -- Euler Taveira EDB https://www.enterprisedb.com/ From 3055de48fe71f47df357114c7a42db05edcdb290 Mon Sep 17 00:00:00 200

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-01-06 Thread Bossart, Nathan
On 12/2/21, 1:34 PM, "Bossart, Nathan" wrote: > On 12/2/21, 9:50 AM, "David Steele" wrote: >> On 12/2/21 12:38, David Steele wrote: >>> On 12/2/21 11:00, Andrew Dunstan wrote: Should we really be getting rid of PostgreSQL::Test::Cluster::backup_fs_hot() ? >>> >>> Agreed, it would b

Re: Skipping logical replication transactions on subscriber side

2022-01-06 Thread Masahiko Sawada
On Wed, Jan 5, 2022 at 12:31 PM Amit Kapila wrote: > > On Mon, Dec 27, 2021 at 9:54 AM Masahiko Sawada wrote: > > > > On Thu, Dec 16, 2021 at 2:42 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Dec 16, 2021 at 2:21 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Dec 16, 2021 at 10:37 A

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-01-06 Thread Euler Taveira
On Thu, Jan 6, 2022, at 9:48 PM, Bossart, Nathan wrote: > After a quick glance, I didn't see an easy way to hold a session open > while the test does other things. If there isn't one, modifying > backup_fs_hot() to work with non-exclusive mode might be more trouble > than it is worth. You can use

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Tom Lane
"Blake, Geoff" writes: > Hope everything is well going into the new year. I'd like to pick this > discussion back up and your thoughts on the patch with the data I posted 2 > weeks prior. Is there more data that would be helpful? Different setup? > Data on older versions of Postgresql to as

RE: Support tab completion for upper character inputs in psql

2022-01-06 Thread tanghy.f...@fujitsu.com
On Thursday, January 6, 2022 11:57 PM, Tom Lane wrote: > > Peter Eisentraut writes: > > So the ordering of the suggested completions is different. I don't know > > offhand how that ordering is determined. Perhaps it's dependent on > > locale, readline version, or operating system. In any case

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-06 Thread Imseih (AWS), Sami
Thanks for the review. I am hesitant to make column name changes for obvious reasons, as it breaks existing tooling. However, I think there is a really good case to change "index_vacuum_count" as the name is confusing. "index_vacuum_cycles_completed" is the name I suggest if we agree to rename.

Re: \dP and \dX use ::regclass without "pg_catalog."

2022-01-06 Thread Justin Pryzby
https://www.postgresql.org/message-id/flat/7ad8cd13-db5b-5cf6-8561-dccad1a934cb%40nttcom.co.jp https://www.postgresql.org/message-id/flat/20210827193151.GN26465%40telsasoft.com On Sat, Aug 28, 2021 at 08:57:32AM -0400, Álvaro Herrera wrote: > On 2021-Aug-27, Justin Pryzby wrote: > > I noticed that

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Andres Freund
Hi, > I landed on the idea of adding some intentional spinlock > contention to src/test/modules/test_shm_mq, which is a prefab test > framework for passing data among multiple worker processes. The > attached quick-hack patch makes it grab and release a spinlock once > per passed message. I wond

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Tom Lane
Andres Freund writes: >> I landed on the idea of adding some intentional spinlock >> contention to src/test/modules/test_shm_mq, which is a prefab test >> framework for passing data among multiple worker processes. The >> attached quick-hack patch makes it grab and release a spinlock once >> per

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Andres Freund
Hi, On 2022-01-06 21:39:57 -0500, Tom Lane wrote: > Andres Freund writes: > > I wonder if this will show the full set of spinlock contention issues - > > isn't > > this only causing contention for one spinlock between two processes? > > I don't think so -- the point of using the "pipelined" var

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Tom Lane
Andres Freund writes: > These separate shm_mq instances forward messages in a circle, > "leader"->worker_1->worker_2->...->"leader". So there isn't a single contended > spinlock, but a bunch of different spinlocks, each with at most two backends > accessing it? No; there's just one spinlock. I'm

Re: [PATCH] Add extra statistics to explain for Nested Loop

2022-01-06 Thread Lukas Fittl
On Sun, Nov 21, 2021 at 8:55 PM Justin Pryzby wrote: > I'm curious to hear what you and others think of the refactoring. > > It'd be nice if there's a good way to add a test case for verbose output > involving parallel workers, but the output is unstable ... > I've reviewed this patch, and it wo

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Alexander Korotkov
,On Thu, Jan 6, 2022 at 4:15 PM Finnerty, Jim wrote: > (Maxim) Re: -- If after upgrade page has no free space for special data, > tuples are >converted to "double xmax" format: xmin became virtual >FrozenTransactionId, xmax occupies the whole 64bit. >Page converted to new

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Andres Freund
On 2022-01-06 22:23:38 -0500, Tom Lane wrote: > No; there's just one spinlock. I'm re-purposing the spinlock that > test_shm_mq uses to protect its setup operations (and thereafter > ignores). Oh, sorry, misread :( > AFAICS the N+1 shm_mq instances don't internally contain > spinlocks; they all

Re: Bugs in pgoutput.c

2022-01-06 Thread Amit Kapila
On Fri, Jan 7, 2022 at 1:28 AM Tom Lane wrote: > > Amit Kapila writes: > > > + * *during* a callback if we do any syscache or table access in the > > + * callback. > > > As we don't take locks on tables, can invalidation events be accepted > > during table access? I could be missing something but

Re: Deduplicate min restart_lsn calculation code

2022-01-06 Thread Bharath Rupireddy
On Thu, Jan 6, 2022 at 11:54 PM Alvaro Herrera wrote: > > On 2022-Jan-06, Bharath Rupireddy wrote: > > > Hi, > > > > It seems like the two functions ReplicationSlotsComputeRequiredLSN and > > ReplicationSlotsComputeLogicalRestartLSN more or less does the same > > thing which makes me optimize (sav

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-06 Thread Takashi Menjo
Hi Justin, Thank you for your build test and comments. The v7 patchset attached to this email fixes the issues you reported. > The cfbot showed issues compiling on linux and windows. > http://cfbot.cputube.org/takashi-menjo.html > > https://cirrus-ci.com/task/6125740327436288 > [02:30:06.538] In

Re: Support tab completion for upper character inputs in psql

2022-01-06 Thread Japin Li
On Fri, 07 Jan 2022 at 10:12, tanghy.f...@fujitsu.com wrote: > On Thursday, January 6, 2022 11:57 PM, Tom Lane wrote: >> >> Peter Eisentraut writes: >> > So the ordering of the suggested completions is different. I don't know >> > offhand how that ordering is determined. Perhaps it's depend

Re: row filtering for logical replication

2022-01-06 Thread Amit Kapila
On Thu, Jan 6, 2022 at 6:42 PM Euler Taveira wrote: > > On Thu, Jan 6, 2022, at 1:18 AM, Amit Kapila wrote: > > On Thu, Jan 6, 2022 at 8:43 AM Peter Smith wrote: > > > > On Wed, Jan 5, 2022 at 9:52 PM Amit Kapila wrote: > > > > > ... > > > > > Another minor comment: > > > +static bool pgoutput_r

  1   2   >