pg_statsinfo - PG15 support?

2022-10-05 Thread Greg Nancarrow
removed and some new fields have been added). I also noticed that PG15 changes how shared libraries must request additional shared memory during initialization and pg_statsinfo source code would need updating for this. Regards, Greg Nancarrow Fujitsu Australia

Re: On login trigger: take three

2022-02-15 Thread Greg Nancarrow
On Mon, Jan 24, 2022 at 1:59 PM Greg Nancarrow wrote: > > I've attached a re-based version (no functional changes from the > previous) to fix cfbot failures. > I've attached a re-based version (no functional changes from the previous) to fix cfbot failures. Regards,

Re: row filtering for logical replication

2022-01-31 Thread Greg Nancarrow
c.test1 WHERE TRUE; (since currently, parentheses are required around the publication WHERE expression) See also the following commit, which specifically added these parentheses and catered for WHERE TRUE: https://www.postgresql.org/message-id/attachment/121245/0005-fixup-Publication-WHERE-condition-support-for-pg_dum.patch Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-31 Thread Greg Nancarrow
(aschemaPubids, puboid)) topmost_relid = ancestor; } Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-31 Thread Greg Nancarrow
I don't believe the calls to MemoryContextSwitchTo() are required here, because within the context switch it's just freeing memory, not allocating it. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-30 Thread Greg Nancarrow
ested: - v15 psql connecting to same and older versions, and using "\dRp+" and "\d " commands - v15 pg_dump, dumping the above definitions from the same or older server versions - Loading dumps from older or same (v15) server version into a v15 server. I did not detect any issues. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-30 Thread Greg Nancarrow
t function. IMO removing "static" makes things worse because now that array gets initialized each call to the function, which is unnecessary. I think it should just be: "static const int map_changetype_pubaction[] = ..." Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-28 Thread Greg Nancarrow
, "\d" by itself doesn't show any row filter information, so I think it should say: Psql commands "\dRp+" and "\d " will display any row filters. Regards, Greg Nancarrow Fujitsu Australia

Re: Fix uninitialized variable access (src/backend/utils/mmgr/freepage.c)

2022-01-27 Thread Greg Nancarrow
make this more obvious. if (btp == NULL) { result->page = NULL; result->found = false; return; } Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-26 Thread Greg Nancarrow
On Thu, Jan 27, 2022 at 4:59 PM Peter Smith wrote: > > On Thu, Jan 27, 2022 at 9:40 AM Greg Nancarrow wrote: > > > > On Wed, Jan 26, 2022 at 2:08 PM houzj.f...@fujitsu.com > > wrote: > > > > > > There was a miss in the posted patc

Re: row filtering for logical replication

2022-01-26 Thread Greg Nancarrow
re required doc/src/sgml/ref/create_subscription.sgml (5) CREATE SUBSCRIPTION BEFORE: + returns false or null will not be published. If the subscription has several AFTER: + evaluates to false or null will not be published. If the subscription has several Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-26 Thread Greg Nancarrow
nt to be: + * - (Var Op Const) Logical-Op (Var Op Const) ? It's not clear what "Bool" means here. (3) check_simple_rowfilter_expr_walker We should say "Built-in functions" instead of "System-functions": + * User-defined functions are not allowed. System-functions that are Regards, Greg Nancarrow Fujitsu Australia

Re: PublicationActions - use bit flags.

2022-01-24 Thread Greg Nancarrow
as pass-by-reference, and I'd tend to stick with pass-by-reference in that case. [1] https://postgr.es/m/OS0PR01MB5716B899A66D2997EF28A1B3945F9%40OS0PR01MB5716.jpnprd01.prod.outlook.com Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-23 Thread Greg Nancarrow
On Mon, Jan 24, 2022 at 5:09 PM Amit Kapila wrote: > > On Mon, Jan 24, 2022 at 10:29 AM Greg Nancarrow wrote: > > > > On Mon, Jan 24, 2022 at 2:47 PM Amit Kapila wrote: > > > > > > > (3) pgoutput_row_filter_exec_expr > > > > pgoutput_row_f

Re: row filtering for logical replication

2022-01-23 Thread Greg Nancarrow
s-ci.com/task/6280873841524736?logs=test_world#L3970 > 2139062143 is 0x7F7F7F7F, so it looks like a value from uninitialized memory (debug build) has been copied into the column, or something similar involving uninitialized memory. The problem is occurring on FreeBSD. I tried using similar build flags as

Re: row filtering for logical replication

2022-01-23 Thread Greg Nancarrow
l" is true, the function always returns false, not the value of "ret". For the current logging code to be correct, and match the function return value, we should be able to change: if (isnull) return false; to: if (isnull) return ret; But regression tests fail when that code change is made (indicating that there are cases when "isnull" is true but the function returns true instead of false). So the current logging code is NOT correct, and needs to be updated as I indicated. Regards, Greg Nancarrow Fujitsu Australia

Re: On login trigger: take three

2022-01-23 Thread Greg Nancarrow
On Mon, Dec 6, 2021 at 12:10 PM Greg Nancarrow wrote: > > I've attached a re-based version (no functional changes from the > previous) to fix cfbot failures. > I've attached a re-based version (no functional changes from the previous) to fix cfbot failures. Regards,

Re: row filtering for logical replication

2022-01-21 Thread Greg Nancarrow
* consumption and CPU cycles. (5) pgoutput_row_filter BEFORE: + * evaluates the row filter for that tuple and return. AFTER: + * evaluate the row filter for that tuple and return. Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2022-01-20 Thread Greg Nancarrow
"clear" in the following comment? (the subtransaction's XID was never being cleared before, just checked against the skipxid, and now that check has been removed) + * ... . Since we don't + * support skipping individual subtransactions we don't clear + * subtransactio

Re: row filtering for logical replication

2022-01-20 Thread Greg Nancarrow
nd was already discussed on another thread ([1]) on which I posted a patch to correct the issue along the same lines that you're suggesting. [1] https://postgr.es/m/CAJcOf-d0%3DvQx1Pzbf%2BLVarywejJFS5W%2BM6uR%2B2d0oeEJ2VQ%2BEw%40mail.gmail.com Regards, Greg Nancarrow Fujitsu Australia

Re: PublicationActions - use bit flags.

2022-01-20 Thread Greg Nancarrow
On Tue, Dec 21, 2021 at 12:55 PM Greg Nancarrow wrote: > > On Tue, Dec 21, 2021 at 11:56 AM Tom Lane wrote: > > > > Removing this is not good: > > > > if (relation->rd_pubactions) > > - { > > pfree(relation-

Re: row filtering for logical replication

2022-01-19 Thread Greg Nancarrow
lt-in data types and non-immutable built-in functions.") AFTER: + errdetail("Expressions only allow columns, constants, built-in operators, built-in data types and immutable built-in functions.") Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2022-01-18 Thread Greg Nancarrow
commits the clearing of subskixpid here, so I suggest the following addition: BEFORE: + * clear subskipxid of pg_subscription. AFTER: + * clear subskipxid of pg_subscription, then commit. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-17 Thread Greg Nancarrow
On Tue, Jan 18, 2022 at 2:31 PM Amit Kapila wrote: > > On Tue, Jan 18, 2022 at 8:41 AM Greg Nancarrow wrote: > > > > On Tue, Jan 18, 2022 at 2:31 AM houzj.f...@fujitsu.com > > wrote: > > > > > > > (2) GetTopMostAncestorInPublication > >

Re: row filtering for logical replication

2022-01-17 Thread Greg Nancarrow
nt ? > Maybe the existing comment should be updated to just spell it out like that: /* * Find the "topmost" ancestor that is in this publication, by getting the * last Oid in the ancestors list which is published by the publication. */ Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Greg Nancarrow
e I think the following test case is misleading and should be removed, because the "1.1" xid value is only regarded as invalid because "1" is an invalid xid (and there's already a test case for a "1" xid) - the fractional part gets thrown away, and doesn't affect the validity here. +ALTER SUBSCRIPTION regress_testsub SKIP (xid = 1.1); Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2022-01-16 Thread Greg Nancarrow
EFORE: + * It is only safe to execute UPDATE/DELETE when all columns referenced in + * the row filters from publications which the relation is in are valid - AFTER: + * It is only safe to execute UPDATE/DELETE when all columns, referenced in + * the row filters from publications which the relation is in, are valid - Regards, Greg Nancarrow Fujitsu Australia

Re: Failed transaction statistics to measure the logical replication progress

2021-12-21 Thread Greg Nancarrow
header should mention something like: "The supplied repWorker statistics are cleared upon return, to assist re-use by the caller." Regards, Greg Nancarrow Fujitsu Australia

Re: PublicationActions - use bit flags.

2021-12-20 Thread Greg Nancarrow
using the rd_pubactions pointer as a boolean flag to indicate whether the publication membership info has been fetched (so the bool flags are valid). I guess you'd need another bool flag if you wanted to make that struct in-line in the relcache entry. Regards, Greg Nancarrow Fujitsu Australia

Re: PublicationActions - use bit flags.

2021-12-20 Thread Greg Nancarrow
sary. I've attached a patch which addresses that and replaces a couple of memcpy()s with struct assignment, as suggested. Regards, Greg Nancarrow Fujitsu Australia get_rel_pubactions_improvement.patch Description: Binary data

Re: PublicationActions - use bit flags.

2021-12-19 Thread Greg Nancarrow
ssary use of a palloc'd tiny struct to return the PublicationActions (which also currently isn't explicitly pfree'd). Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-12-19 Thread Greg Nancarrow
re sense to: 1) Disallow different WHERE clauses on the same table, for different pubactions. 2) If only INSERTs are being published, allow any column in the WHERE clause, otherwise (as for UPDATE and DELETE) restrict the referenced columns to be part of the replica identity or primary key. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-12-17 Thread Greg Nancarrow
On Fri, Dec 17, 2021 at 7:20 PM Ajin Cherian wrote: > > On Fri, Dec 17, 2021 at 5:46 PM Greg Nancarrow wrote: > > > So using the v47 patch-set, I still find that the UPDATE above results in > > publication of an INSERT of (2,1), rather than an UPDATE of (1,1) to (2,1). >

Re: row filtering for logical replication

2021-12-16 Thread Greg Nancarrow
me that the old (current) row was previously UPDATEd (and published, or not published, according to the filter applicable to UPDATE), but this is not necessarily the case. Or am I missing something? [1] https://postgr.es/m/CAJcOf-dz0srExG0NPPgXh5X8eL2uxk7C=czogtbf8cnqoru...@mail.gmail.com Regards, Greg Nancarrow Fujitsu Australia

Re: Failed transaction statistics to measure the logical replication progress

2021-12-15 Thread Greg Nancarrow
arer, and the stats more meaningful? [1] https://www.postgresql.org/message-id/flat/cahut+pt39pbqs0sxt9rmm89ayizoq0kw46yzskkzwk8z5ho...@mail.gmail.com Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-15 Thread Greg Nancarrow
ase usage on other similar entries? (e.g. "Two phase commit") src/include/catalog/pg_subscription.h (3) + bool subdisableonerr; /* True if apply errors should disable the + * subscription upon error */ The comment should just say "True if occurren

Re: row filtering for logical replication

2021-12-15 Thread Greg Nancarrow
> I was thinking such subscription requests could be rejected by the server, based on the subscriber version and whether the publications use filtering etc. Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-12-14 Thread Greg Nancarrow
d a way to allow users to specify only XID that has failed. > Yes, I agree that would be better. If you didn't do that, I think you'd need to queue the XIDs to be skipped (rather than locking). Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-12-14 Thread Greg Nancarrow
al "*/" (so the asterisks align). Also, should say "... treated the same". /* + * If the publication is FOR ALL TABLES then it is treated same as if this + * table has no filters (even if for some other publication it does). + */ Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-12-13 Thread Greg Nancarrow
ssue, right? If a different xid to skip is specified while the worker is currently skipping a transaction, should that even be allowed? Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-12-12 Thread Greg Nancarrow
ip the transaction in question +# by ALTER SUBSCRIPTION ... SKIP. Then, check if logical replication can continue +# working by inserting $nonconflict_data on the publisher. (ii) BEFORE: +# will conflict with the data replicated from publisher later. AFTER: +# will conflict with the data replicated later from the publisher. Regards, Greg Nancarrow Fujitsu Australia

Re: Fix typos - "an" instead of "a"

2021-12-08 Thread Greg Nancarrow
ot;. I found an article that > seems to further support this since it both sounds like a vowel (oh) and is > also a letter (oh). > > https://www.grammar.com/a-vs-an-when-to-use > What about the "-" before the "o"? Wouldn't it be read as "dash o" or "minus o"? This would mean "a" is correct, not "an", IMHO. Regards, Greg Nancarrow Fujitsu Australia

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-08 Thread Greg Nancarrow
t scenario, but I needed to reduce the amount of inserted data (so reduced 200 to 2) due to disk space. I then consistently get an error like the following: postgres=# alter database test set tablespace pg_default; ERROR: could not create file "pg_tblspc/16385/PG_15_202111301/16386/36395": File exists (this only happens when the patch is used) Regards, Greg Nancarrow Fujitsu Australia

Re: Alter all tables in schema owner fix

2021-12-07 Thread Greg Nancarrow
hange "Create Publication" to "CREATE PUBLICATION". Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-06 Thread Greg Nancarrow
urred for more than X minutes) or, similarly, perhaps if some threshold has been reached (e.g. same error has occurred more than X times), but I think that this was previously suggested by Peter Smith and the idea wasn't looked upon all that favorably? Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-05 Thread Greg Nancarrow
ion subscription \"%s\" will be disabled due to error: %s", +MySubscription->name, edata->message)); Perhaps a similar error message could be logged prior to EmitErrorReport()? e.g. "logical replication subscription \"%s\" will be disabled due to an error" Regards, Greg Nancarrow Fujitsu Australia

Re: On login trigger: take three

2021-12-05 Thread Greg Nancarrow
On Tue, Nov 16, 2021 at 4:38 PM Greg Nancarrow wrote: > > I've attached an updated patch with these changes. > I've attached a re-based version (no functional changes from the previous) to fix cfbot failures. Regards, Greg Nancarrow Fujitsu Australia v23-0001-Add-a-new-lo

Re: row filtering for logical replication

2021-12-02 Thread Greg Nancarrow
ii) In the above case (where values and nulls are palloc'd), shouldn't the values and nulls be pfree()d at the end of the function? 0005 src/backend/utils/cache/relcache.c (1) RelationGetInvalRowFilterCol Shouldn't "rfnode" be pfree()d after use? Regards, Greg Nancarrow Fujitsu Australia

Re: Alter all tables in schema owner fix

2021-12-02 Thread Greg Nancarrow
er_arg(newOwnerId)) as: + if (!superuser_arg(newOwnerId) && IsSchemaPublication(form->oid)) Regards, Greg Nancarrow Fujitsu Australia

Re: Data is copied twice when specifying both child and parent table in publication

2021-12-01 Thread Greg Nancarrow
On Thu, Dec 2, 2021 at 1:48 PM Greg Nancarrow wrote: > > If you updated my original description to say "(instead of just the > individual partitions)", it would imply the same I think. > But I don't mind if you want to explicitly state both cases to make it clear. >

Re: Data is copied twice when specifying both child and parent table in publication

2021-12-01 Thread Greg Nancarrow
ould imply the same I think. But I don't mind if you want to explicitly state both cases to make it clear. Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-01 Thread Greg Nancarrow
scription + worker detects non-transient errors (e.g. duplication error) + that require user intervention to resolve. What do you think? Regards, Greg Nancarrow Fujitsu Australia

Re: Data is copied twice when specifying both child and parent table in publication

2021-12-01 Thread Greg Nancarrow
we should back-patch this but I am slightly worried ... I'd be in favor of back-patching this. Regards, Greg Nancarrow Fujitsu Australia

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-30 Thread Greg Nancarrow
en critical system index 2662 pg_dumpall: error: pg_dump failed on database "test2", exiting Hopefully the above example will help in tracking down the cause. Regards, Greg Nancarrow Fujitsu Australia

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-30 Thread Greg Nancarrow
teDBRelInfo" needs to be added to src/tools/pgindent/typedefs.list src/bin/pg_rewind/parsexlog.c (3) Include additional header file It seems that the following additional header file is not needed to compile the source file: +#include "utils/relmapper.h" Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-29 Thread Greg Nancarrow
of pg_subscription. Regards, Greg Nancarrow Fujitsu Australia

Re: Correct handling of blank/commented lines in PSQL interactive-mode history

2021-11-29 Thread Greg Nancarrow
On Tue, Nov 30, 2021 at 12:15 PM Greg Nancarrow wrote: > > Yeah, sorry, looks like it could be a Gmail issue for me. > When I alternatively downloaded your patches from the pgsql-hackers > archive, they're in Unix format, as you say. > After a bit of investigation, it seems th

Re: Correct handling of blank/commented lines in PSQL interactive-mode history

2021-11-29 Thread Greg Nancarrow
On Tue, Nov 30, 2021 at 11:08 AM Tom Lane wrote: > > Greg Nancarrow writes: > > (BTW, the patches are in Windows CRLF format, so on Linux at least I > > needed to convert them using dos2unix so they'd apply using Git) > > Hmm. Applying "od -c" to the copy

Re: Correct handling of blank/commented lines in PSQL interactive-mode history

2021-11-29 Thread Greg Nancarrow
effort. The patches LGTM. Thanks for working on this. (BTW, the patches are in Windows CRLF format, so on Linux at least I needed to convert them using dos2unix so they'd apply using Git) Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-28 Thread Greg Nancarrow
ls for UPDATE) and so has to assume the old tuple doesn't exist on the subscriber, hence the UPDATE ends up doing an INSERT. INow if the use of RI cols were enforced for the insert filter case, we'd properly know the answer as to whether the old row value had been published and it would have correctly performed an UPDATE instead of an INSERT in this case. Thoughts? Regards, Greg Nancarrow Fujitsu Australia

Re: Windows build warnings

2021-11-25 Thread Greg Nancarrow
gres developers on Windows for cases unrelated to the use of PG_USED_FOR_ASSERTS_ONLY. I agree with your proposal to apply your patch to remove PG_USED_FOR_ASSERTS_ONLY where it's no longer applicable. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-25 Thread Greg Nancarrow
filter columns are not part of the REPLICA IDENTITY"))); should be: + errdetail("Not all row filter columns are part of the REPLICA IDENTITY"))); Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-25 Thread Greg Nancarrow
uot; in the following comment, to match the struct member name: * subworker is the hash table of PgStat_StatSubWorkerEntry which stores Otherwise, the patch LGTM. Regards, Greg Nancarrow Fujitsu Australia

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-24 Thread Greg Nancarrow
uot;, &relfilenode, forkname); how about using the following instead in this case: + nmatch = sscanf(filename, "%u_%4s", &relfilenode, forkname); ? (even if there were > 4 chars after the underscore, it would still match and InvalidOid would be returned because nmatch==2) Regards, Greg Nancarrow Fujitsu Australia

Re: Windows build warnings

2021-11-23 Thread Greg Nancarrow
USED_FOR_ASSERTS_ONLY in the same style. > Isn't "[[maybe_unused]]" only supported for MS C++ (not C)? I'm using Visual Studio 17, and I get nothing but a syntax error if trying to use it in C code, whereas it works if I rename the same source file to have a ".cpp" extension (but even then I need to use the "/std:c++17" compiler flag) Regards, Greg Nancarrow Fujitsu Australia

Re: Failed transaction statistics to measure the logical replication progress

2021-11-22 Thread Greg Nancarrow
olved without intervention, logical replication will get stuck in a loop retrying and the last error will occur again and again, hence the count of how many times that has happened. Maybe there's not much benefit in counting different errors prior to the last error? Regards, Greg Nancarrow Fujitsu Australia

Windows build warnings

2021-11-22 Thread Greg Nancarrow
re also reported here: [1] I've attached a patch to fix these warnings. (Note that currently PG_USED_FOR_ASSERTS_ONLY is defined as the unused attribute, which is only supported by GCC) [1]: https://www.postgresql.org/message-id/cah2-wznwwu+9on9nzcnztk7ua238mctgpxyr1ty7u_msn5z...@mail.gmail

Re: row filtering for logical replication

2021-11-21 Thread Greg Nancarrow
not freed: src/backend/replication/pgoutput/pgoutput.c + if (rfnodes) + { + list_free(rfnodes); + rfnodes = NIL; + } Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-19 Thread Greg Nancarrow
h the most latest info. > Couldn't the code block in pgstat_recv_subworker_error() that increments error_count just compare the new msg timestamp against the existing first_error_time and last_error_time and, based on the result, update those if required? Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Greg Nancarrow
> Oops, I missed that too. So at worst, vacuum will clean it up in the out-of-order SUBSCRIPTIONPURGE,SUBWORKERERROR case. But I still think the current code may not correctly handle first_error_time/last_error_time timestamps if out-of-order SUBWORKERERROR messages occur, right? Regard

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Greg Nancarrow
ence messaging due to UDP similarly mean that "first_error_time" and "last_error_time" may not be currently handled correctly? Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-18 Thread Greg Nancarrow
Relation relation; + Node *whereClause; } PublicationRelInfo; It appears that this new member is not actually required, as the relid can be simply obtained from the existing "relation" member - using the RelationGetRelid() macro. Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Greg Nancarrow
uses "null" but the "command" column description uses "NULL". I think "NULL" should be used for consistency. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-18 Thread Greg Nancarrow
nk a test needs to be added similar to the customers+countries example that Tomas gave (where there is a single subscription to multiple publications of the same table, each of which has a row-filter). Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-17 Thread Greg Nancarrow
rr" instead of "disable_on_error". Also "disable_on_err" appears in a couple of the test case comments. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-17 Thread Greg Nancarrow
TextDatumGetCString(rfdatum)); rfnodes = lappend(rfnodes, rfnode); + MemoryContextSwitchTo(oldctx); (these changes are needed in addition to the fixes I posted on this thread for the crash problem that was previously reported) Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-16 Thread Greg Nancarrow
the 0006 patch (when I remove that patch, the problem doesn't occur). Still needs investigation. Regards, Greg Nancarrow Fujitsu Australia

Re: row filtering for logical replication

2021-11-16 Thread Greg Nancarrow
ossible memory leak issue that may need further investigation. Regards, Greg Nancarrow

Re: On login trigger: take three

2021-11-15 Thread Greg Nancarrow
t being said, that's still not a very readable name, maybe someone else has > an even better suggestion? > Changed to "dathasloginevt", as suggested. I've attached an updated patch with these changes. I also noticed one of the Windows-based cfbots was failing with an "SS

Re: row filtering for logical replication

2021-11-15 Thread Greg Nancarrow
entry->rowfilter_valid = true; @@ -1881,7 +1879,7 @@ rel_sync_cache_relation_cb(Datum arg, Oid relid) } if (entry->exprstate != NULL) { -free(entry->exprstate); +pfree(entry->exprstate); entry->exprstate = NULL; } } Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-14 Thread Greg Nancarrow
line (did you run pg_indent?) src/include/pgstat.h (3) Remove PgStat_StatSubWorkerEntry.dbid? The "dbid" member of the new PgStat_StatSubWorkerEntry struct doesn't seem to be used, so I think it should be removed. (I could remove it and everything builds OK and tests pass). Regards, Greg Nancarrow Fujitsu Australia

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-11 Thread Greg Nancarrow
DisableSubscriptionOnError() instead of the memory-context, and the existing MemoryContextSwitch() and CopyErrorData() calls could be removed from it. AFAICS, applying (3) and (4) above would make the code a lot cleaner. Regards, Greg Nancarrow Fujitsu Australia

Re: On login trigger: take three

2021-11-11 Thread Greg Nancarrow
; That being said, that's still not a very readable name, maybe someone else has > an even better suggestion? > Yes you're right, in this case I was wrongly treating "evt" as an abbreviation for "event". I agree "dathasloginevt" would be better. Regards, Greg Nancarrow Fujitsu Australia

Re: Data is copied twice when specifying both child and parent table in publication

2021-11-11 Thread Greg Nancarrow
e the potential for inconsistent data resulting in this case, unless say, the subscriber "child tables" are first truncated on the refresh, if they are in fact partitions of the root, and then the table sync publishes the existing data via the root) Regards, Greg Nancarrow Fujitsu Australia

Re: On login trigger: take three

2021-11-10 Thread Greg Nancarrow
and remove its restoration in pg_dump output, since it's not needed (as in v20 patch) - Some tidying of the updates to the event_trigger tests and capitalization of the test SQL Regards, Greg Nancarrow Fujitsu Australia v21-0001-Add-a-new-login-event-and-login-event-trigger-support.patch Description: Binary data

Re: On login trigger: take three

2021-11-09 Thread Greg Nancarrow
On Fri, Nov 5, 2021 at 3:03 PM Greg Nancarrow wrote: > > +1 > The arguments given are pretty convincing IMHO, and I agree that the > additions made in the v20 patch are not a good idea, and are not needed. > If there are no objections, I plan to reinstate the previous v1

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-09 Thread Greg Nancarrow
On Wed, Nov 10, 2021 at 3:22 PM Greg Nancarrow wrote: > > I had a look at this patch and have a couple of initial review > comments for some issues I spotted: > Incidentally, I found that the v3 patch only applies after the skip xid v20 patch [1] has been applied.

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-09 Thread Greg Nancarrow
tSwitchTo(ecxt); + } I'm guessing it was intended to do the "MemoryContextSwitch(ecxt);" before re-throwing (?), but it's not really clear, as in the 1st and 3rd cases, the DisableSubscriptionOnError() calls anyway immediately switch the memory context to cctx. Regards, Greg Nancarrow Fujitsu Australia

Re: Failed transaction statistics to measure the logical replication progress

2021-11-08 Thread Greg Nancarrow
the underlying latest v20 patch. Regards, Greg Nancarrow Fujitsu Australia

Re: Failed transaction statistics to measure the logical replication progress

2021-11-08 Thread Greg Nancarrow
atch was posted a day ago, it looks like your patch needs to be rebased against that (as it currently applies on top of the v19 version only). Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-07 Thread Greg Nancarrow
t;stats_reset" is currently documented as the last column of the "pg_stat_subscription_workers" view - but it's actually no longer included in the view. (5) src/tools/pgindent/typedefs.list The following current entries are bogus: PgStat_MsgSubWorkerErrorPurge PgStat_MsgSubWorkerPurge The following entry is missing: PgStat_MsgSubscriptionPurge Regards, Greg Nancarrow Fujitsu Australia

Re: On login trigger: take three

2021-11-04 Thread Greg Nancarrow
#x27;m overly paranoid, but adding a backdoor of sorts for a situation > which > really shouldn't happen doesn't seem like a good idea. > > +1 The arguments given are pretty convincing IMHO, and I agree that the additions made in the v20 patch are not a good idea, and are not needed. Regards, Greg Nancarrow Fujitsu Australia

Re: Data is copied twice when specifying both child and parent table in publication

2021-11-04 Thread Greg Nancarrow
On Thu, Nov 4, 2021 at 7:10 PM Amit Kapila wrote: > > On Thu, Nov 4, 2021 at 12:23 PM Greg Nancarrow wrote: > > > > On Thu, Nov 4, 2021 at 3:13 PM Amit Kapila wrote: > > > > > > On further thinking about this, I think we should define the behavior > >

Re: Data is copied twice when specifying both child and parent table in publication

2021-11-03 Thread Greg Nancarrow
How about: - If "publish_via_partition_root" is true for a publication, then data is replicated to the table with the same name as the root (i.e. partitioned) table in the publisher. - If "publish_via_partition_root" is false (the default) for a publication, then data is replicat

Re: Consider parallel for lateral subqueries with limit

2021-11-03 Thread Greg Nancarrow
On Thu, Nov 4, 2021 at 12:49 AM James Coleman wrote: > > Greg, > > Do you believe this is now ready for committer? > The patch LGTM. I have set the status to "Ready for Committer". Regards, Greg Nancarrow Fujitsu Australia

Re: Failed transaction statistics to measure the logical replication progress

2021-11-03 Thread Greg Nancarrow
nction. (11) update_apply_change_size() Shouldn't this function be declared static? (12) stream_write_change() + streamed_entry->xact_size = streamed_entry->xact_size + total_len; /* update */ could be simply written as: + streamed_entry->xact_size += total_len; /* update */ Regards, Greg Nancarrow Fujitsu Australia

Re: Skipping logical replication transactions on subscriber side

2021-11-01 Thread Greg Nancarrow
e that this refers to "tablesync", so perhaps add " (tablesync)" to the end of this sentence, or similar. Regards, Greg Nancarrow Fujitsu Australia

Re: Added schema level support for publication.

2021-11-01 Thread Greg Nancarrow
r the "objtype" column, the type "schema" does not seem specific enough; maybe that should be instead named "all-tables-in-schema" or similar. Regards, Greg Nancarrow Fujitsu Australia

Re: Added schema level support for publication.

2021-10-31 Thread Greg Nancarrow
tion structure and members in SQL, without having to piece the information together from the other pg_publication_* tables. Personally I don't think it is necessary to additionally display all tables in the schema (that information can be retrieved by pg_tables or information_schema.tables, if need

Skip vacuum log report code in lazy_scan_heap() if possible

2021-10-29 Thread Greg Nancarrow
skipping this code if ultimately nothing is going to be logged (and I found that even for a tiny database, a VACUUM may execute this code hundreds of times). I have attached a simple patch for this. Regards, Greg Nancarrow Fujitsu Australia v1-0001-Skip-vacuum-log-report-processing-in-lazy_scan_heap

  1   2   3   4   5   >