Re: Skipping logical replication transactions on subscriber side

2021-12-17 Thread Peter Eisentraut
On 13.12.21 04:12, Greg Nancarrow wrote: (ii) "Setting -1 means to reset the transaction ID" Shouldn't it be explained what resetting actually does and when it can be, or is needed to be, done? Isn't it automatically reset? I notice that negative values (other than -1) seem to be regarded as val

Re: psql format output

2021-12-17 Thread Peter Eisentraut
On 15.12.21 20:58, Florian Koch wrote: I realized that the output of "\df+ func_name" has a formatting problem when a lot of arguments are used. The field 'Arguments data types' gets very long and destroys the whole formatting in the console. The field 'Source code' is most of the time multi-li

Re: Adding CI to our tree

2021-12-17 Thread Peter Eisentraut
On 13.12.21 22:12, Andres Freund wrote: Attached is an updated version of the CI patches. An example of a test run with the attached version of this https://cirrus-ci.com/build/6501998521483264 + only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-o

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-12-17 Thread Peter Geoghegan
2 today? SKIP_PAGES_THRESHOLD hasn't changed since commit bf136cf6e3, shortly after the original visibility map implementation was committed in 2009. The idea that it helps us to advance relfrozenxid outside of aggressive VACUUMs (per commit message from bf136cf6e3) seems like it might no longer matter with the patch -- because now we won't ever set a page all-visible but not all-frozen. Plus the idea that we need to do all this work just to get readahead from the OS seems...questionable. -- Peter Geoghegan

Re: row filtering for logical replication

2021-12-19 Thread Peter Smith
> I think currently there could be a problem with user perceptions. IMO a user would be mostly interested in predictability and getting results that are intuitive. So, even if all strange results can (after careful examination) be after-the-fact explained away as being "correct" according to a spec, I don't think that is going to make any difference. e.g. regardless of correctness, even if it just "appeared" to give unexpected results then a user may just decide that row-filtering is not worth their confusion... Perhaps there is a slightly dumbed-down RF design that can still be useful, but which can give much more comfort to the user because the replica will be more like what they were expecting? -- Kind Regards, Peter Smith. Fujitsu Australia

PublicationActions - use bit flags.

2021-12-19 Thread Peter Smith
simplifies assignment and checking of said flags. PSA a small patch for this. Thoughts? -- Kind Regards, Peter Smith. Fujitsu Australia v1-0001-PublicationActions-use-bit-flags.patch Description: Binary data

Re: row filtering for logical replication

2021-12-19 Thread Peter Smith
7 SET ALL TABLES IN SCHEMA > testpub_rf_myschema1, TABLE testpub_rf_myschema1.testpub_rf_tb16; > +RESET client_min_messages; > Fixed in v48 [1] -- [1] https://www.postgresql.org/message-id/CAHut%2BPuHz1oFM7oaiHeqxMQqd0L70bV_hT7u_mDf3b8As5kwig%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2021-12-19 Thread Peter Smith
+ and \dRp+). Currently, there is no pg_get_pubdef function (analogous to the index's pg_get_indexdef) so that would need to be written from scratch. So I feel this is a good feature, but it could be implemented as an independent patch in another thread. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-12-20 Thread Peter Eisentraut
On 15.12.21 14:15, Gilles Darold wrote: Le 15/12/2021 à 13:41, Peter Eisentraut a écrit : On 03.08.21 19:10, Tom Lane wrote: Gilles Darold writes: Sorry I have missed that, but I'm fine with this implemenation so let's keep the v6 version of the patch and drop this one. Pu

Use JOIN USING aliases in ruleutils.c

2021-12-20 Thread Peter Eisentraut
only commented out some code that could be removed, not actually removed it.From da15f7571f43e8eef279c3e66530d67ae9b5ef7f Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 20 Dec 2021 11:37:15 +0100 Subject: [PATCH v1] Use JOIN USING aliases in ruleutils.c When reverse-compiling a query

Re: Column Filtering in Logical Replication

2021-12-20 Thread Peter Eisentraut
On 17.12.21 22:07, Alvaro Herrera wrote: So I've been thinking about this as a "security" item (you can see my comments to that effect sprinkled all over this thread), in the sense that if a publication "hides" some column, then the replica just won't get access to it. But in reality that's mist

Re: PublicationActions - use bit flags.

2021-12-20 Thread Peter Eisentraut
On 20.12.21 01:18, Peter Smith wrote: For some reason the current HEAD PublicationActions is a struct of boolean representing combinations of the 4 different "publication actions". I felt it is more natural to implement boolean flag combinations using a bitmask instead of a struct of

Re: Getting rid of regression test input/ and output/ files

2021-12-20 Thread Peter Eisentraut
On 19.12.21 00:53, Tom Lane wrote: 2. Export the values from pg_regress as environment variables, and then add a way for the test scripts to read those variables. I was a bit surprised to realize that we didn't have any way to do that already --- psql has \setenv, so why did we never invent \gete

Re: Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-20 Thread Peter Eisentraut
On 18.12.21 21:57, Chapman Flack wrote: I sometimes do some testing as nobody, on a distro where getpwent(nobody)->pw_dir is a directory that nobody can't write. So I end up setting $HOME to a directory that, um, is writable. When I start psql, strace shows $HOME being honored when looking for .

Re: sequences vs. synchronous replication

2021-12-20 Thread Peter Eisentraut
On 18.12.21 22:48, Tomas Vondra wrote: What do you mean by "not caching unused sequence numbers"? Reducing SEQ_LOG_VALS to 1, i.e. WAL-logging every sequence increment? That'd work, but I wonder how significant the impact will be. It'd bet it hurts the patch adding logical decoding of sequence

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-20 Thread Peter Geoghegan
On Mon, Nov 29, 2021 at 6:51 PM Peter Geoghegan wrote: > Attached is a WIP patch doing this. This has bitrot, so I attach v2, mostly just to keep the CFTester status green. The only real change is one minor simplification to how we set everything up, inside heap_vacuum_rel(). -- Pe

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Peter Geoghegan
what you've called options #2 and #3, but those options still seem unappealing to me. -- Peter Geoghegan

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Peter Geoghegan
bulkdelete scan). Note also that we have to do something called backtracking in btvacuumpage(), which you've ignored -- that's another reasonably common way that we'll end up scanning a page twice. But that probably should just be ignored -- it's too narrow a case to be worth caring about. -- Peter Geoghegan

Re: row filtering for logical replication

2021-12-20 Thread Peter Smith
0001, 0002, and 0005. I > would > probably merge 0004 because it is just isolated changes. > > [1] https://en.wikipedia.org/wiki/Type_conversion > > Thanks for all the suggested fixes. Next, I plan to post a new v51* patch set which will be 1. Take your "fixes" patches, and wherever possible just merge them back into the main patches. 2. Merge the resulting main patches according to Amit's advice. -- Kind Regards, Peter Smith. Fujitsu Australia.

Re: do only critical work during single-user vacuum?

2021-12-20 Thread Peter Geoghegan
g like the attached as part of a more worked out solution (assuming it didn't fully remove single user mode from the equation, which would be better still). -- Peter Geoghegan v1-0001-Always-trigger-failsafe-in-single-user-mode.patch Description: Binary data

Re: do only critical work during single-user vacuum?

2021-12-20 Thread Peter Geoghegan
-- not just the table (or tables) whose relfrozenxid is very old. That's still naive, but much less naive than simply telling users to VACUUM the whole database in single user mode while vacuuming indexes, etc. -- Peter Geoghegan

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-12-20 Thread Peter Geoghegan
ght now (without the patch) by "escalating" a regular VACUUM to an aggressive one. But the cases that we still expect an aggressive VACUUM (with the patch) are the cases where there is zero chance of that happening. Almost by definition. > Given the opportunistic freezing, that's true but I'm concerned > whether opportunistic freezing always works well on all tables since > freezing tuples is not 0 cost. That is the big question for this patch. -- Peter Geoghegan

Re: do only critical work during single-user vacuum?

2021-12-21 Thread Peter Geoghegan
op-level command (that doesn't accept any tables). The only reason to have this is for scenarios where the user is already in a tough spot with wraparound failure, like that client of yours. Nobody wants to force the failsafe for one specific table. It's not general purpose, at all, and shouldn't claim to be. -- Peter Geoghegan

Re: row filtering for logical replication

2021-12-21 Thread Peter Smith
together with the one explanatory comment, but if you still want the #define moved please confirm and I can do it in a future version. > I think we can now merge 0001, 0002, and 0005. We are still evaluating > the performance for 0003, so it is better to keep it separate. We can > t

Re: row filtering for logical replication

2021-12-21 Thread Peter Smith
[2 Amit] https://www.postgresql.org/message-id/CAA4eK1KwoA5k8v9z9e4ZPN_X%3D1GAmQmsWyauFwZpKiSHqy6eZA%40mail.gmail.com [3 Ajin] https://www.postgresql.org/message-id/CAFPTHDbfpPNh3GLGjySS%3DAuRfbQPQFNvfiyG1GDQW2kz1yT7Og%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2021-12-21 Thread Peter Smith
t; + > > Like: > /* Add qualifications, if available */ > if (pri->whereClause != NULL) > { > /* Set up a ParseState to parse with */ > pstate = make_parsestate(NULL); > > /* > * Get the transformed WHERE clause, of boolean type, with necessary > * collation information. > */ > whereclause = GetTransformedWhereClause(pstate, pri, true); > > /* > * Walk the parse-tree of this publication row filter expression and > * throw an error if anything not permitted or unexpected is > * encountered. > */ > rowfilter_walker(whereclause, targetrel); > values[Anum_pg_publication_rel_prqual - 1] = > CStringGetTextDatum(nodeToString(whereclause)); > } > else > nulls[Anum_pg_publication_rel_prqual - 1] = true; > Fixed in v52-0001 [1] as suggested. -- [1] https://www.postgresql.org/message-id/CAHut%2BPs3BvAqcNXmMMRBUjOe3GWor0d7r%2BmPxxtzMhYEf59t_Q%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

fix crash with Python 3.11

2021-12-22 Thread Peter Eisentraut
crashes in the plpython_transaction test. The fix is that we need to catch the PostgreSQL error and turn it into a Python exception, like we do for other places where plpy.* methods call into PostgreSQL internals.From 31bbd62f43ceb0542e0a311dc28704fd702caeb3 Mon Sep 17 00:00:00 2001 From: Peter

Re: track_io_timing default setting

2021-12-22 Thread Peter Geoghegan
clock_gettime() -- it's just for getting the time, not measuring elapsed time. It seems reasonable to suppose that an operating system that offers a facility for measuring elapsed time won't have horrible performance problems. clock_gettime() first appeared in POSIX almost 30 years ago. -- Peter Geoghegan

Re: do only critical work during single-user vacuum?

2021-12-22 Thread Peter Geoghegan
A is tired and stressed. Something that makes generic choices, that are not way too conservative, but also don't risk making the problem worse instead of better. -- Peter Geoghegan

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-22 Thread Peter Geoghegan
fferent (confusingly similar) metrics shown in each variant of the instrumentation (a problem that I'm trying to permanently avoid by unifying everything). While things have improved a lot here recently, I don't think that things have fully settled yet -- the output will probably ch

Re: row filtering for logical replication

2021-12-23 Thread Peter Smith
On Thu, Dec 23, 2021 at 7:23 PM Peter Smith wrote: > > Here is the v54* patch set: > > Main changes from v53* are > 1. All files of all three patches have been pgindented. > 2. Another review comment is addressed > > ~~ > > Details > === > > v51-0001 (

Inconsistent ellipsis in regression test error message?

2021-12-23 Thread Peter Smith
mmand? Thanks for any ideas! -- [2] https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.157408 Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2021-12-23 Thread Peter Smith
n the WHERE clause. AFTER: + A nullable column in the WHERE clause could cause the + expression to evaluate to false. To avoid unexpected results, any possible + null values should be accounted for. Thoughts? -- Kind Regards, Peter Smith. Fujitsu Australia

Add Boolean node

2021-12-27 Thread Peter Eisentraut
cast, and internal Boolean values in DDL commands were usually represented by Integer nodes. This takes the place of both of these uses, making the intent clearer and having some amount of type safety.From 4e1ef56b5443fa11d981eb6e407dfc7c244dc60e Mon Sep 17 00:00:00 2001 From: Peter Eisentraut

Re: psql - add SHOW_ALL_RESULTS option

2021-12-27 Thread Peter Eisentraut
On 23.12.21 12:40, Fabien COELHO wrote: In [0], it was reported that certain replication commands result in infinite loops because of faulty error handling.  This still happens. I wrote a test for it, attached here.  (I threw in a few more basic tests, just to have some more coverage that was

Re: sequences vs. synchronous replication

2021-12-27 Thread Peter Eisentraut
On 24.12.21 09:04, Kyotaro Horiguchi wrote: Still, as Fujii-san concerns, I'm afraid that some people may suffer the degradation the patch causes. I wonder it is acceptable to get back the previous behavior by exposing SEQ_LOG_VALS itself or a boolean to do that, as a 'not-recommended-to-use' va

Re: Add Boolean node

2021-12-27 Thread Peter Eisentraut
On 27.12.21 14:15, Ashutosh Bapat wrote: That looks like a good change. I wonder what motivates that now? Why wasn't it added when the usages grew? Are there more Boolean usages planned? Mainly, I was looking at Integer/makeInteger() and noticed that most uses of those weren't actually integer

parse/analyze API refactoring

2021-12-28 Thread Peter Eisentraut
#x27;t make sense to begin with and should be in pg_rewrite_query().From 9a72ceadebd6b9309fe0157c1cd181b29ee8bd52 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Tue, 28 Dec 2021 09:01:45 +0100 Subject: [PATCH v1 1/3] Parse/analyze function renaming There are three parallel ways to call parse/analyze: with fix

Re: PublicationActions - use bit flags.

2021-12-28 Thread Peter Smith
here is v2 of my original bit-flag replacement for the PublicationActions struct. This version introduces one more bit flag for the relcache status, and by doing so means all that code for Relation cache PublicationActions pointers and pallocs and context switches can just disappear... -- Kind

Re: Foreign key joins revisited

2021-12-29 Thread Peter Eisentraut
On 28.12.21 20:45, Vik Fearing wrote: I don't particularly like this whole idea anyway, but if we're going to have it, I would suggest JOIN ... USING KEY ... since USING currently requires a parenthesized list, that shouldn't create any ambiguity. In the 1990s, there were some SQL drafts

Re: Converting WAL to SQL

2021-12-29 Thread Peter Eisentraut
On 29.12.21 07:18, rajesh singarapu wrote: I am wondering if we have a mechanism to convert WAL records to SQL statements. I am able to use logical decoders like wal2json or test_decoding for converting WAL to readable format, but I am looking for a way to convert WAL to sql statements. Usi

Re: UNIQUE null treatment option

2021-12-29 Thread Peter Eisentraut
Here is a rebased version of this patch. On 27.08.21 14:38, Peter Eisentraut wrote: The SQL standard has been ambiguous about whether null values in unique constraints should be considered equal or not.  Different implementations have different behaviors.  In the SQL:202x draft, this has been

Re: automatically generating node support functions

2021-12-29 Thread Peter Eisentraut
el.deFrom e2c08d8b793200a07b8fe5ae85dd23f401ddcef1 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 29 Dec 2021 12:00:41 +0100 Subject: [PATCH v3] Automatically generate node support functions Add a script to automatically generate the node support functions (copy, equal, out, and re

Re: PublicationActions - use bit flags.

2021-12-29 Thread Peter Smith
On Thu, Dec 30, 2021 at 3:30 AM Justin Pryzby wrote: > > On Mon, Dec 20, 2021 at 11:18:41AM +1100, Peter Smith wrote: > > For some reason the current HEAD PublicationActions is a struct of > > boolean representing combinations of the 4 different "publication > > a

Re: Add Boolean node

2021-12-30 Thread Peter Eisentraut
On 29.12.21 21:32, Andres Freund wrote: On 2021-12-27 09:53:32 -0500, Tom Lane wrote: Didn't really read the patch in any detail, but I did have one idea: I think that the different things-that-used-to-be-Value-nodes ought to use different field names, say ival, rval, bval, sval not just "val".

Re: Non-decimal integer literals

2021-12-30 Thread Peter Eisentraut
o yet. This affects patches 0004 and 0005. As mentioned before, patches 0006 and 0007 are more feature previews at this point. On 01.12.21 16:47, Peter Eisentraut wrote: On 25.11.21 18:51, John Naylor wrote: If we're going to change the comment anyway, "the parser" sounds more

Re: ICU for global collation

2021-12-30 Thread Peter Eisentraut
s to check that colcollate and colctype have to be the same for ICU, so it's overall cleaner.From 4eb9fbac238c1abf481fa43431ecc22e782a5290 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 30 Dec 2021 12:47:24 +0100 Subject: [PATCH v3] Add option to use ICU as global collation pro

Re: Add Boolean node

2022-01-03 Thread Peter Eisentraut
On 27.12.21 10:02, Peter Eisentraut wrote: This patch adds a new node type Boolean, to go alongside the "value" nodes Integer, Float, String, etc.  This seems appropriate given that Boolean values are a fundamental part of the system and are used a lot. Before, SQL-level Boolean

Re: Add Boolean node

2022-01-03 Thread Peter Eisentraut
On 03.01.22 12:04, Peter Eisentraut wrote: On 27.12.21 10:02, Peter Eisentraut wrote: This patch adds a new node type Boolean, to go alongside the "value" nodes Integer, Float, String, etc.  This seems appropriate given that Boolean values are a fundamental part of the system and

Re: Synchronizing slots from primary to standby

2022-01-03 Thread Peter Eisentraut
Here is an updated patch to fix some build failures. No feature changes. On 14.12.21 23:12, Peter Eisentraut wrote: On 31.10.21 11:08, Peter Eisentraut wrote: I want to reactivate $subject.  I took Petr Jelinek's patch from [0], rebased it, added a bit of testing.  It basically works, b

Re: psql - add SHOW_ALL_RESULTS option

2022-01-03 Thread Peter Eisentraut
On 23.12.21 12:40, Fabien COELHO wrote: This is also a feature/bug of libpq which happens to be hidden by PQexec: when one command crashes PQgetResult actually returns *2* results. First one with the FATAL message, second one when libpq figures out that the connection was lost with the second m

Re: row filtering for logical replication

2022-01-03 Thread Peter Smith
} > What about changing this if statement like following: > if (entry->exprstate_valid) > return; > Modified in v58 [1] as suggested -- [1] https://www.postgresql.org/message-id/CAHut%2BPvkswkGLqzYo7z9rwOoDeLtUk0PEha8kppNvZts0h22Hw%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2022-01-03 Thread Peter Smith
On Mon, Dec 27, 2021 at 9:57 AM Euler Taveira wrote: > > On Sun, Dec 26, 2021, at 1:09 PM, Alvaro Herrera wrote: > > On 2021-Dec-26, Euler Taveira wrote: > > > On Sat, Dec 25, 2021, at 1:20 AM, Amit Kapila wrote: > > > On Fri, Dec 24, 2021 at 11:04 AM Peter Smith &

Re: Inconsistent ellipsis in regression test error message?

2022-01-03 Thread Peter Smith
On Tue, Dec 28, 2021 at 4:34 AM Tom Lane wrote: > > Peter Smith writes: > > The most recent cfbot run for a patch I am interested in has failed a > > newly added regression test. > > Please see http://cfbot.cputube.org/ for 36/2906 > > The failure logs [2] are

Re: row filtering for logical replication

2022-01-03 Thread Peter Smith
On Fri, Dec 31, 2021 at 12:39 AM houzj.f...@fujitsu.com wrote: > > On Wed, Dec 29, 2021 11:16 AM Tang, Haiying wrote: > > On Mon, Dec 27, 2021 9:16 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Thur, Dec 23, 2021 4:28 PM Peter Smith wrote

Re: [PATCH] Allow multiple recursive self-references

2022-01-04 Thread Peter Eisentraut
On 21.09.21 13:35, Denis Hirn wrote: Also, currently a query like this works [...] but this doesn't: WITH RECURSIVE t(n) AS ( SELECT n+1 FROM t WHERE n < 100 UNION ALL VALUES (1) ) SELECT sum(n) FROM t; With your patch, the second should also work, so let's show some tests for that a

Re: [PATCH] Allow multiple recursive self-references

2022-01-04 Thread Peter Eisentraut
I have some separate questions on the executor changes. Basically, this seems the right direction, but I wonder if some things could be clarified. I wonder why in ExecWorkTableScan() and ExecReScanWorkTableScan() you call tuplestore_copy_read_pointer() instead of just tuplestore_select_read_p

Re: ICU for global collation

2022-01-04 Thread Peter Eisentraut
On 04.01.22 03:21, Julien Rouhaud wrote: @@ -2774,6 +2776,7 @@ dumpDatabase(Archive *fout) appendPQExpBuffer(dbQry, "SELECT tableoid, oid, datname, " "(%s datdba) AS dba, " "pg_enco

Re: row filtering for logical replication

2022-01-04 Thread Peter Smith
d in row filters from all the publications + * the relation is in are part of replica identity, or the publication + * actions do not include UPDATE and DELETE. + */ Some minor rewording of the comment: "true" --> "True". "part of replica identity" --> "part of the replica identity" "UPDATE and DELETE" --> "UPDATE or DELETE" -- Kind Regards, Peter Smith. Fujitsu Australia.

Re: row filtering for logical replication

2022-01-04 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:34 PM Peter Smith wrote: > > I have reviewed again the source code for v58-0001. > > Below are my review comments. > > Actually, I intend to fix most of these myself for v59*, so this post > is just for records. > &g

Re: Proposal: remove obsolete hot-standby testing infrastructure

2022-01-05 Thread Peter Eisentraut
On 03.01.22 22:50, Tom Lane wrote: The attached proposed patch removes some ancient infrastructure for manually testing hot standby. I doubt anyone has used this in years, because AFAICS there is nothing here that's not done better by the src/test/recovery TAP tests. (Or if there is, we ought t

Re: SQL:2011 application time

2022-01-05 Thread Peter Eisentraut
On 21.11.21 02:51, Paul A Jungwirth wrote: Here are updated patches. They are rebased and clean up some of my TODOs. This patch set looks very interesting. It's also very big, so it's difficult to see how to get a handle on it. I did a pass through it to see if there were any obvious architec

Re: row filtering for logical replication

2022-01-05 Thread Peter Smith
FYI - v58 is currently known to be broken due to a recent commit [1]. I plan to post a v59* later today to address this as well as other recent review comments. -- [1] https://github.com/postgres/postgres/commit/6ce16088bfed97f982f66a9dc17b8364df289e4d Kind Regards, Peter Smith. Fujitsu

Remove trailing comma from enums

2022-01-05 Thread Peter Smith
I saw one (and then went looking and found some more) enum with a trailing comma. These are quite rare in the PG src, so I doubt they are intentional. PSA a patch to remove the trailing commas for all that I found. -- Kind Regards, Peter Smith. Fujitsu Australia v1-0001-Remove-trailing

Re: Remove trailing comma from enums

2022-01-05 Thread Peter Smith
On Thu, Jan 6, 2022 at 12:23 PM Tom Lane wrote: > > Thomas Munro writes: > > On Thu, Jan 6, 2022 at 12:56 PM Peter Smith wrote: > >> These are quite rare in the PG src, so I doubt they are intentional. > >> PSA a patch to remove the trailing commas for all that I

Re: row filtering for logical replication

2022-01-05 Thread Peter Smith
hat no information was accidentally lost along the way. We understand that prior to the final commit this will all need to be fused together just like you are suggesting. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2022-01-05 Thread Peter Smith
ntly no typedef for the enum ReorderBufferChangeType. Of course, it is easy to add a typedef and then this 'enum' is not needed in the signature, but I wasn't sure if adding a new typedef strictly belonged as part of this Row-Filter patch or not. -- Kind Regards. Peter Smith. Fujitsu Australia.

Re: Support tab completion for upper character inputs in psql

2022-01-05 Thread Peter Eisentraut
On 10.09.21 15:50, tanghy.f...@fujitsu.com wrote: (A test case for the enum case should be doable easily.) Test added. The enum test is failing on *some* platforms: t/010_tab_completion.pl .. 26/? # Failed test 'complete enum values' # at t/010_tab_completion.pl line 211. # Actual output

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2022-01-06 Thread Peter Geoghegan
ompared to any other update_vacuum_error_info() call (or any other call that doesn't set a saved_err_info). -- Peter Geoghegan v3-0001-Unify-VACUUM-VERBOSE-and-log_autovacuum_min_durat.patch Description: Binary data

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Peter Geoghegan
the first place (in code like hio.c and the FSM) to make all this work. [1] https://www.postgresql.org/message-id/CAH2-Wz%3DzEV4y_wxh-A_EvKxeAoCMdquYMHABEh_kZO1rk3a-gw%40mail.gmail.com -- Peter Geoghegan

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
bactions can be used for row filtering. > Fixed in v59* [1] -- [1] https://www.postgresql.org/message-id/CAHut%2BPsiw9fbOUTpCMWirut1ZD5hbWk8_U9tZya4mG-YK%2Bfq8g%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:26 PM Amit Kapila wrote: > > On Tue, Jan 4, 2022 at 12:15 PM Peter Smith wrote: > > > > On Fri, Dec 31, 2021 at 12:39 AM houzj.f...@fujitsu.com > > wrote: > > > > 3) v55-0002 > > > > +static bool pgoutput_row_filter_u

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:34 PM Peter Smith wrote: > > I have reviewed again the source code for v58-0001. > > Below are my review comments. > > Actually, I intend to fix most of these myself for v59*, so this post > is just for records. > &g

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
-- [1] https://www.postgresql.org/message-id/CAHut%2BPsiw9fbOUTpCMWirut1ZD5hbWk8_U9tZya4mG-YK%2Bfq8g%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 1:05 PM wangw.f...@fujitsu.com wrote: > > On Thu, Jan 4, 2022 at 00:54 PM Peter Smith wrote: > > Modified in v58 [1] as suggested > Thanks for updating the patches. > A few comments about v58-0001 and v58-0002. > > v58-0001 > 1. > How about m

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 12:54 PM Robert Haas wrote: > On Fri, Dec 17, 2021 at 9:30 PM Peter Geoghegan wrote: > > Can we fully get rid of vacuum_freeze_table_age? Maybe even get rid of > > vacuum_freeze_min_age, too? Freezing tuples is a maintenance task for > > physical block

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 2:45 PM Peter Geoghegan wrote: > But the "freeze early" heuristics work a bit like that anyway. We > won't freeze all the tuples on a whole heap page early if we won't > otherwise set the heap page to all-visible (not all-frozen) in the

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 1:05 PM wangw.f...@fujitsu.com wrote: > > On Thu, Jan 4, 2022 at 00:54 PM Peter Smith wrote: > > Modified in v58 [1] as suggested > Thanks for updating the patches. > A few comments about v58-0001 and v58-0002. > > v58-0001 > 1. > How about m

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
a single statement so the parentheses are not needed here. ~~ 6. src/include/replication/logicalproto.h +extern void logicalrep_write_update_cached(StringInfo out, TransactionId xid, Relation rel, +TupleTableSlot *oldtuple, TupleTableSlot *newtuple, +bool binary); This extern seems unused ??? Kind Regards, Peter Smith. Fujitsu Australia

Re: preserve timestamps when installing headers

2022-01-06 Thread Peter Eisentraut
On 04.01.22 22:21, Tom Lane wrote: However, there's another problem with using INSTALL_DATA as a solution to this issue: why would you expect that to preserve timestamps? install-sh won't. I see that /usr/bin/install (which configure picks on my RHEL box) won't preserve them by default, but it h

Re: Column Filtering in Logical Replication

2022-01-07 Thread Peter Eisentraut
I think this is getting pretty good now. I like the overall behavior now. Some details: There are still a few references to "filter", but I see most of the patch now uses column list or something. Maybe do another cleanup pass before finalizing the patch. doc/src/sgml/catalogs.sgml needs to b

Re: ICU for global collation

2022-01-07 Thread Peter Eisentraut
On 04.01.22 17:03, Peter Eisentraut wrote: There are really a lot of places with this new code.  Maybe it could be some new function/macro to wrap that for the normal case (e.g. not formatting.c)? Right, we could just put this into pg_newlocale_from_collation(), but the comment there says

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Peter Eisentraut
On 07.01.22 06:18, Fujii Masao wrote: On 2022/01/06 19:24, Simon Riggs wrote: On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: Your opinions are very much welcome! This is a review of the Int64 options patch, "v6-0001-Add-64-bit-GUCs-for-xids.patch" Do we really need to support both int32

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-07 Thread Peter Geoghegan
arguments I've made here. > I agree that it's OK for this to become a purely backstop mechanism > ... but again, I think that the design of such backstop mechanisms > should be done as carefully as we know how, because users seem to hit > the backstop all the time. We want it to be made of, you know, nylon > twine, rather than, say, sharp nails. :-) Absolutely. But if autovacuum can only ever run due to age(relfrozenxid) reaching autovacuum_freeze_max_age, then I can't see a downside. Again, the v5-0005-*patch needs to meet the standard that I've laid out. If it doesn't then I've messed up already. -- Peter Geoghegan

Re: SQL:2011 application time

2022-01-10 Thread Peter Eisentraut
On 06.01.22 06:44, Paul A Jungwirth wrote: I didn't follow why indexes would have periods, for example, the new period field in IndexStmt. Is that explained anywhere? When you create a primary key or a unique constraint (which are backed by a unique index), you can give a period name to make

small bug in ecpg unicode identifier error handling

2022-01-10 Thread Peter Eisentraut
I think this patch is necessary: diff --git a/src/interfaces/ecpg/preproc/pgc.l b/src/interfaces/ecpg/preproc/pgc.l index 07fee80a9c..3529b2ea86 100644 --- a/src/interfaces/ecpg/preproc/pgc.l +++ b/src/interfaces/ecpg/preproc/pgc.l @@ -753,7 +753,7 @@ cppline {space}*#([^i][

Re: row filtering for logical replication

2022-01-10 Thread Peter Smith
unexpected object depending on column: publication of table t1 in publication p1 test_pub=# ~~ But the message looks OK. What exactly was your expectation for this review comment? -- Kind Regards, Peter Smith. Fujitsu Australia.

Re: row filtering for logical replication

2022-01-10 Thread Peter Smith
Col-Filtering". Thoughts? -- Kind Regards, Peter Smith. Fujitsu Australia

Re: ICU for global collation

2022-01-11 Thread Peter Eisentraut
On 10.01.22 07:00, Julien Rouhaud wrote: And then I changed in varstr_cmp(): if (collid != DEFAULT_COLLATION_OID) mylocale = pg_newlocale_from_collation(collid); to just mylocale = pg_newlocale_from_collation(collid); I find that the \timing results are indist

Re: ICU for global collation

2022-01-11 Thread Peter Eisentraut
On 07.01.22 10:03, Julien Rouhaud wrote: I changed the datcollate, datctype, and the new daticucoll fields to type text (from name). That way, the daticucoll field can be set to null if it's not applicable. Also, the limit of 63 characters can actually be a problem if you want to use some com

Re: ICU for global collation

2022-01-11 Thread Peter Eisentraut
On 10.01.22 12:49, Daniel Verite wrote: With the current patch, it's not possible, AFAICS, because the user can't tell that the collation is non-deterministic. Presumably this would require another option to CREATE DATABASE and another column to store that bit of information. Adding this would

Re: Time to drop plpython2?

2022-01-11 Thread Peter Eisentraut
On 15.11.21 19:52, Peter Eisentraut wrote: I think we should just write to the build farm owners, we plan to drop python2 support in, say, 60 days, please update your setup to use python3 or disable python support. This discussion stalled. I think we should do *something* for PostgreSQL 15

Re: do only critical work during single-user vacuum?

2022-01-11 Thread Peter Geoghegan
ool that allows the DBA to do this without excessive downtime. The GetNewTransactionId() WARNINGs ought to be changed to reference VACUUM LIMIT. (You probably just didn't get around to that in this POC, but couldn't hurt to remind you.) -- Peter Geoghegan

Re: row filtering for logical replication

2022-01-11 Thread Peter Smith
sCache(rftuple); + } Those 3 assignments to the context.pubviaroot/parentid/relid can be moved to be inside the if (!rfisnull) block, because IIUC they don't get used otherwise. Or, maybe better to just leave as-is; I am not sure what is best. Please consider. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: row filtering for logical replication

2022-01-11 Thread Peter Smith
typedefs.list - ReorderBufferChangeType Actually, the typedef for ReorderBufferChangeType was added in the 62-0001, so this typedef change should've been done in patch 0001 and it can be removed from patch 0002 -- [1] https://www.postgresql.org/message-id/CAHut%2BPucFM3Bt-gaTT7Pr-Y_x%2

Re: Time to drop plpython2?

2022-01-11 Thread Peter Eisentraut
On 11.01.22 17:06, Tom Lane wrote: Nonetheless, we need to make a recommendation to the buildfarm owners about what's the minimum python3 version we intend to support going forward. Do we want to just set it at 3.6, with the expectation that the meson move will happen before too long? Well, th

Re: small bug in ecpg unicode identifier error handling

2022-01-12 Thread Peter Eisentraut
On 10.01.22 14:14, Peter Eisentraut wrote: I think this patch is necessary: diff --git a/src/interfaces/ecpg/preproc/pgc.l b/src/interfaces/ecpg/preproc/pgc.l index 07fee80a9c..3529b2ea86 100644 --- a/src/interfaces/ecpg/preproc/pgc.l +++ b/src/interfaces/ecpg/preproc/pgc.l @@ -753,7 +753,7

Re: Column Filtering in Logical Replication

2022-01-12 Thread Peter Eisentraut
On 12.01.22 01:41, Alvaro Herrera wrote: Therefore, I propose to add an index on pg_publication_rel.prpubid. That seems very reasonable.

Re: Postgres Replication from windows to linux

2022-01-12 Thread Peter Eisentraut
On 10.01.22 19:56, rajesh singarapu wrote: I am trying to migrate my postgres to linux, as we are moving away from windows. I am trying both dump/restore  and logical decoding, but people are not happy with performance. Is there a way/tooling I can use around WAL shipping/physical replication h

Re: disfavoring unparameterized nested loops

2022-01-12 Thread Peter Geoghegan
project? I was rather hoping that you'd get back to it, FWIW. -- Peter Geoghegan

<    4   5   6   7   8   9   10   11   12   13   >