speed up a logical replica setup

2022-02-21 Thread Euler Taveira
criber and pg_createsubscriber are names that I thought but I'm not excited about it. DOCUMENTATION It is available and describes this tool. TESTS Basic tests are included. It requires some tests to exercise this tool. Comments? [1] https://github.com/2ndQuadrant/pglogical -- Euler Tave

Re: row filtering for logical replication

2022-02-21 Thread Euler Taveira
e that has been discussed for 4 years [1]. [1] https://www.postgresql.org/message-id/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2022-02-21 Thread Euler Taveira
mmand via replication protocol. The disadvantages are: (a) it could duplicate code and (b) it might require maintenance if new options are added to the BASE_BACKUP command. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Euler Taveira
it. > Hmm, now that you say it, maybe the ^PGXS regex should be case-insensitive, > if pgxs can be written in e.g. lower case? Makefile variable names are case-sensitive. You cannot write pgxs or PgXs; it should be PGXS. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Frontend error logging style

2022-02-23 Thread Euler Taveira
ms while I was working on pg_subscriber [1]. Your proposal looks good to me. [1] https://postgr.es/m/5ac50071-f2ed-4ace-a8fd-b892cffd33eb%40www.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logical decoding and replication of sequences

2022-02-23 Thread Euler Taveira
nse to replicate other relkinds, but maybe DDL? Materialized view? As you mentioned DDL, maybe we can use the CREATE PUBLICATION syntax to select which DDL commands we want to replicate. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logical decoding and replication of sequences

2022-02-23 Thread Euler Taveira
On Wed, Feb 23, 2022, at 4:18 PM, Tomas Vondra wrote: > On 2/23/22 18:33, Euler Taveira wrote: > > On Wed, Feb 23, 2022, at 1:07 PM, Tomas Vondra wrote: > >> Maybe, but I don't think it's very common to have that many > >> schemas added to the same publicatio

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-25 Thread Euler Taveira
dn't find it. https://postgr.es/m/20180629.173418.190173462.horiguchi.kyot...@lab.ntt.co.jp -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Commitfest manager for 2022-03

2022-02-27 Thread Euler Taveira
On Sat, Feb 26, 2022, at 9:37 PM, Justin Pryzby wrote: > |https://commitfest.postgresql.org/37/2906/ > |Row filtering for logical replication > If I'm not wrong, this is merged and should be closed? I think Amit forgot to mark it as committed. Done. -- Euler Tavei

Re: logical replication restrictions

2022-02-28 Thread Euler Taveira
On Wed, Sep 22, 2021, at 1:57 PM, Euler Taveira wrote: > On Wed, Sep 22, 2021, at 1:18 AM, Amit Kapila wrote: >> On Tue, Sep 21, 2021 at 4:21 PM Marcos Pegoraro wrote: >>> No, I´m talking about that configuration you can have on standby servers >>> recovery_min_apply_

Re: logical replication restrictions

2022-03-01 Thread Euler Taveira
On Tue, Mar 1, 2022, at 3:27 AM, osumi.takami...@fujitsu.com wrote: > $ git am v1-0001-Time-delayed-logical-replication-subscriber.patch I generally use -3 to fall back on 3-way merge. Doesn't it work for you? -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2022-03-02 Thread Euler Taveira
5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500); 159 \dRp+ testpub5 IIRC this test was written before adding the row filter information into the psql. We could add \d+ testpub_rf_tbl3 before and after the modification. > 2) There are no pg_dump tests. WFM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2022-03-03 Thread Euler Taveira
On Thu, Mar 3, 2022, at 7:47 AM, Amit Kapila wrote: > LGTM. I'll push this tomorrow unless Tomas or Euler feels otherwise. Sounds good to me. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Add the replication origin name and commit-LSN to logical replication worker errcontext

2022-03-04 Thread Euler Taveira
.e., > pg_16395) and the next LSN of the commit LSN (i.e., > LSN 0/14C0379). You could also add: After that the replication can be resumed by ALTER SUBSCRIPTION ... ENABLE. Let's provide complete instructions. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: PG DOCS - logical replication filtering

2022-03-10 Thread Euler Taveira
robably reduce the number of rows in the example but I didn't bother to remove them. It seems we can remove some sentences from the CREATE PUBLICATION because we have a new section that explains all of it. I think the link that was added by this patch is sufficient. -- Euler Taveira EDB htt

Re: logical replication restrictions

2022-03-20 Thread Euler Taveira
On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: > Long time, no patch. Here it is. I will provide documentation in the next > version. I would appreciate some feedback. This patch is broken since commit 705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33. I rebased it. I added documentatio

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Euler Taveira
on/t/029_disable_on_error.pl | 94 -- src/test/subscription/t/029_on_error.pl | 183 +++ It seems you are removing a test for 705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33. I should also name 029_on_error.pl to something else such as 030_skip_lsn.pl or a generic name 030_skip_option.pl. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Skipping logical replication transactions on subscriber side

2022-03-21 Thread Euler Taveira
0_skip_option.pl. > > > > As explained in my previous email, I don't think any change is > required for this comment but do let me know if you still think so? Oh, sorry about the noise. I saw mixed tests between the 2 new features and I was confused if it was intentional or not. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logical replication restrictions

2022-03-21 Thread Euler Taveira
On Mon, Mar 21, 2022, at 10:04 PM, Andres Freund wrote: > On 2022-03-20 21:40:40 -0300, Euler Taveira wrote: > > On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: > > > Long time, no patch. Here it is. I will provide documentation in the next > > > version. I wou

Re: logical replication restrictions

2022-03-23 Thread Euler Taveira
On Mon, Mar 21, 2022, at 10:09 PM, Euler Taveira wrote: > On Mon, Mar 21, 2022, at 10:04 PM, Andres Freund wrote: >> On 2022-03-20 21:40:40 -0300, Euler Taveira wrote: >> > On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: >> > > Long time, no patch. Here it is

Re: Logical archiving

2020-12-04 Thread Euler Taveira
ur issue, we should improve the logical decoding mechanism. There was a discussion about allowing logical decoding on the replica that would probably help your use case a lot. -- Euler Taveira http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Logical archiving

2020-12-04 Thread Euler Taveira
eplication slots and replication origin on failover-candidate subscribers should encourage users to adopt such solutions. [1] https://www.postgresql.org/message-id/CAGRY4nx0-ZVnFJV5749QCqwmqBMkjQpcFkYY56a9U6Vf%2Bf7-7Q%40mail.gmail.com [2] https://wiki.postgresql.org/wiki/Logical_replicat

Re: [PATCH] pg_hba.conf error messages for logical replication connections

2021-02-17 Thread Euler Taveira
ond. I would say "it does not match logical replication connections". It seems sufficient to inform the reader that he/she cannot use records with "replication" to match logical replication connections. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Catalog version access

2021-02-21 Thread Euler Taveira
-21 20:59:06-03 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-02-22 Thread Euler Taveira
should be allowed as it helps > power users to create more complex replications if they need. Yes, you can. I have to check if this user-defined operator could possibly break the replication. I will make sure to include a test covering this case too. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [PATCH] pg_hba.conf error messages for logical replication connections

2021-02-22 Thread Euler Taveira
On Sat, Feb 20, 2021, at 7:33 AM, Amit Kapila wrote: > I have used a bit of different wording here to make things clear. > > Let me know what you think of the attached? WFM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-02-26 Thread Euler Taveira
On Mon, Feb 22, 2021, at 9:28 AM, Euler Taveira wrote: > On Mon, Feb 22, 2021, at 7:45 AM, Önder Kalacı wrote: >> Thanks for working on this. I did some review and testing, please see my >> comments below. > I appreciate your review. I'm working on a new patch set and ex

Re: proposal: psql –help reflecting service or URI usage

2021-03-01 Thread Euler Taveira
* Connection options could be moved to the top (maybe after General options) if we consider that it is more important than the other sections (you cannot probably execute psql without using a connection parameter in production). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Identify missing publications from publisher while create/alter subscription.

2021-03-02 Thread Euler Taveira
The other alternative is to document that non-existent publication names can be in the subscription catalog and it is ignored while executing SUBSCRIPTION commands. You could possibly propose a NOTICE/WARNING that informs the user that the SUBSCRIPTION command contains non-existent publication. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: a verbose option for autovacuum

2021-03-08 Thread Euler Taveira
adding a new parameter controlling that. Perfect is the enemy of the good. Let start with this piece of information. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: cleanup temporary files after crash

2021-03-09 Thread Euler Taveira
-- > Michael That was my fault. Editor automatically added #. I'm not sure Tomas will include the tests. If so. the terminology should be adjusted too. +++ b/src/test/recovery/t/022_crash_temp_files.pl @@ -0,0 +1,194 @@ +# +# Test cleanup of temporary files after a crash. s/cleanup/remove/ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-10 Thread Euler Taveira
publications. I agree that "f1 is not a table" is a confusing message at first because foreign table has "table" as description. Maybe if we apply the negation in both messages it would be clear (using the same wording as system tables). ERROR: "f1" is a foreign

Re: cleanup temporary files after crash

2021-03-16 Thread Euler Taveira
lignment while using Vim (that has 8 spaces as default), for example. For now, I will add autocmd BufRead,BufNewFile postgresql.conf* setlocal ts=8 to my .vimrc. We should probably fix some settings that are misaligned such as parallel_setup_cost and shared_preload_libraries. The parameters timezon

cleanup temporary files after crash

2020-10-28 Thread Euler Taveira
Euler Taveira http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From bcd5dcc00cade3a80ee83f88b15f980ac7cdd0f5 Mon Sep 17 00:00:00 2001 From: Euler Taveira Date: Mon, 25 May 2020 00:08:20 -0300 Subject: [PATCH] Control temporary file

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-30 Thread Euler Taveira
rmsg. +$node->append_conf('postgresql.conf', "logging_collector= 'on'"); +$node->append_conf('postgresql.conf', "log_connections= 'on'"); booleans don't need quotes. -- Euler Taveira http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-31 Thread Euler Taveira
On Sat, 31 Oct 2020 at 00:34, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Fri, Oct 30, 2020 at 6:35 PM Euler Taveira > wrote: > > > > + appendStringInfo(&logmsg, "replication "); > > + > > + appendStringInfo(&am

Re: cleanup temporary files after crash

2020-10-31 Thread Euler Taveira
not providing a GUC at all or provide it in the developer section. I've never heard someone saying that they use those temporary files to investigate an issue. Regarding a crash, all information is already available and temporary files don't provide extra details. This new GUC is just

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-04 Thread Euler Taveira
;t be test failures due to different LANG. [1] https://www.postgresql.org/docs/current/nls-programmer.html [2] https://www.postgresql.org/list/pgsql-translators/ [3] https://git.postgresql.org/gitweb/?p=pgtranslation/messages.git;a=summary -- Euler Taveira http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: redundant error messages

2020-11-05 Thread Euler Taveira
{ pg_log_error("database \"%s\": %s", dbname, PQerrorMessage(conn)); exit_nicely(1); } and remove the redundant 'could not connect to database %s' from scripts/common.c. -- Euler Taveira http://w

recovery_target immediate timestamp

2020-11-11 Thread Euler Taveira
0028; timestamp Wed Nov 11 17:37:47 2020; tli 1; prev tli 1; fpw true; xid 0:519; oid 24576; multi 1; offset 0; oldest xid 501 in DB 1; oldest multi 1 in DB 1; oldest/newest commit timestamp xid: 0/0; oldest running xid 519; online Regards, -- Euler Taveira http://www.2ndQuadrant.c

Re: recovery_target immediate timestamp

2020-11-12 Thread Euler Taveira
On Wed, 11 Nov 2020 at 22:40, Fujii Masao wrote: > > On 2020/11/12 6:00, Euler Taveira wrote: > > > The first patch adds a new message that prints the latest completed > checkpoint > > when the consistent state is reached. > > I'm not sure how useful this in

Re: Any objections to implementing LogicalDecodeMessageCB for pgoutput?

2020-11-24 Thread Euler Taveira
ng to this patch but when/if this patch is committed, I will submit a patch to filter messages by prefix. wal2json has a similar (filter-msg-prefixes / add-msg-prefixes) feature and it is useful for cases where you are handling multiple output plugins like wal2json and pgoutput. The idea i

Re: cleanup temporary files after crash

2020-11-26 Thread Euler Taveira
C. However, if we decided to go with the GUC, default behavior should be remove the temporary files after the crash. Regards, -- Euler Taveira http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: row filtering for logical replication

2021-01-31 Thread Euler Taveira
w.postgresql.org/message-id/CA%2BHiwqG3Jz-cRS%3D4gqXmZDjDAi%3D%3D19GvrFCCqAawwHcOCEn4fQ%40mail.gmail.com -- Euler Taveira EnterpriseDB: https://www.enterprisedb.com/ From 78aa13f958d883f52ef0a9796536adf06cd58273 Mon Sep 17 00:00:00 2001 From: Euler Taveira Date: Mon, 18 Jan 2021 11:13:23 -0300 Subj

Re: row filtering for logical replication

2021-02-01 Thread Euler Taveira
in mind, if the expression evaluates to NULL, it returns false and the row won't be replicated. After the commit 3696a600e2, the last patch does not apply cleanly. I'm attaching another version to address the documentation issues. -- Euler Taveira EnterpriseDB: https://www.enterprised

Re: a verbose option for autovacuum

2021-02-01 Thread Euler Taveira
d execution time runs less than l_a_m_d, the output should be discarded. I don't have a strong opinion about this parameter name but I think your suggestion (log_autovaccum_verbosity) is easier to guess what this parameter is for. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [PATCH] Doc: improve documentation of oid columns that can be zero. (correct version)

2021-02-01 Thread Euler Taveira
On Sun, Jan 31, 2021, at 10:27 AM, Joel Jacobson wrote: > Here comes the patch again, now including data. Joel, register this patch into the next CF [1] so we don't lose track of it. [1] https://commitfest.postgresql.org/32/ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Typo in tablesync comment

2021-02-02 Thread Euler Taveira
"The state tracking the progress of the relation synchronization is > additionally stored in shared memory, with SYNCWAIT and CATCHUP only > appearing in memory." WFM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [PATCH] Doc: improve documentation of oid columns that can be zero. (correct version)

2021-02-02 Thread Euler Taveira
because when I try to re-register it says it's taken. > > Not sure who I should ask for help. Anyone? You should probably email: webmaster (at) postgresql (dot) org -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-02-02 Thread Euler Taveira
is not * enough, because of qual expression dependencies. */ I will post a new patch set later. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Can we have a new SQL callable function to get Postmaster PID?

2021-02-03 Thread Euler Taveira
d_file('postmaster.pid'), '\d+'))[1]; regexp_match -- 13496 (1 row) While investigating an issue, you are probably interested in a backend PID or one of the auxiliary processes. In both cases, it is easier to obtain the PIDs. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: pg_replication_origin_drop API potential race condition

2021-02-05 Thread Euler Taveira
o go back and forth. I saw a few occurrences of this pattern in the source code and IMO it could be used when it is not obvious what that value means. Booleans are easier to figure out, however, sometimes integer and text are not. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: pg_replication_origin_drop API potential race condition

2021-02-08 Thread Euler Taveira
On Mon, Feb 8, 2021, at 3:23 AM, Amit Kapila wrote: > Fixed the problem as mentioned above in the attached. This new version looks good to me. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Clean up code

2021-02-09 Thread Euler Taveira
il.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: O(n) tasks cause lengthy startups and checkpoints

2021-12-01 Thread Euler Taveira
about items (1), (2) and (4), we could probably have an option to create some ephemeral logical decoding files into ramdisk (similar to statistics directory). I wouldn't like to hijack this thread but this proposal could alleviate the possible issues that you pointed out. If people are interested in this proposal, I can start a new thread about it. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-12-02 Thread Euler Taveira
e caches keyed and > separated by the 4x different pubactions. row filter is not applied for TRUNCATEs so it is just 3 operations. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-12-06 Thread Euler Taveira
On Mon, Dec 6, 2021, at 3:35 AM, Dilip Kumar wrote: > On Mon, Dec 6, 2021 at 6:49 AM Euler Taveira wrote: > > > > On Fri, Dec 3, 2021, at 8:12 PM, Euler Taveira wrote: > > > > PS> I will update the commit message in the next version. I barely changed > >

Re: row filtering for logical replication

2021-12-06 Thread Euler Taveira
thout > which the main patch is incomplete and buggy w.r.t replica identity. IMO we should merge sub-features as soon as we reach consensus. Every new sub-feature breaks comments, tests and documentation if you want to remove or rearrange patches. It seems I misread 0005. I agree that it is im

Re: Confused comment about drop replica identity index

2021-12-16 Thread Euler Taveira
ition. We could save a syscall and provide a test for indisreplident there. If this restriction is undesirable, we should at least document this choice and probably emit a WARNING for DROP INDEX. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Confused comment about drop replica identity index

2021-12-20 Thread Euler Taveira
g worked > on, either. So I'd like to apply and back-patch the attached, as a > first step, to fix the inconsistency. > What do you think about the attached patch? It forbids the DROP INDEX. We might add a detail message but I didn't in this patch. -- Euler Taveira EDB https://

Re: relcache not invalidated when ADD PRIMARY KEY USING INDEX

2021-12-20 Thread Euler Taveira
Good catch. It seems you can simplify your checking indexForm->indisprimary directly, no? Why did you add new tests for test_decoding? I think the TAP tests alone are fine. BTW, this test is similar to publisher3/subscriber3. Isn't it better to use the same pub/sub to reduce the test execu

Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary

2021-12-23 Thread Euler Taveira
IMO this issue wouldn't occur if you have a monitoring system and alerts and someone to keep an eye on it. If the disk full situation was caused by a failed archive command or a disconnected standby, it is easy to figure out; the fix is simple. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: correct the sizes of values and nulls arrays in pg_control_checkpoint

2021-12-23 Thread Euler Taveira
ldn't be useful for such case. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Delay the variable initialization in get_rel_sync_entry

2021-12-23 Thread Euler Taveira
gt; a > un-published table in which case the cost is noticeable. > > --12.83%--pgoutput_change > |--11.84%--get_rel_sync_entry > |--4.76%--get_rel_relispartition > |--4.70%--get_rel_relkind Good catch. WFM. Deferring variable initialization close to its first use is good practice. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-12-26 Thread Euler Taveira
esults, any > > possible > > + null values should be accounted for. > > > > Your suggested wording sounds reasonable to me. Euler, others, any thoughts? +1. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-12-26 Thread Euler Taveira
On Sun, Dec 26, 2021, at 1:09 PM, Alvaro Herrera wrote: > On 2021-Dec-26, Euler Taveira wrote: > > > On Sat, Dec 25, 2021, at 1:20 AM, Amit Kapila wrote: > > > On Fri, Dec 24, 2021 at 11:04 AM Peter Smith > > > wrote: > > > > > > > > So, IMO

Re: row filtering for logical replication

2022-01-06 Thread Euler Taveira
ORDER_BUFFER_CHANGE_UPDATE and REORDER_BUFFER_CHANGE_DELETE are the first 3 values from the enum, otherwise, it breaks rfnodes and no_filters in pgoutput_row_filter(). I suggest a separate enum that contains only these 3 values. enum RowFilterPublishAction { PUBLISH_ACTION_INSERT, PUBLISH_ACTION_UPDATE, PUBLISH_ACTION_DELETE }; -- Euler Taveira EDB https://www.enterprisedb.com/

fix libpq comment

2022-01-06 Thread Euler Taveira
Hi, While checking the PQping code I noticed that pg_ctl does not rely on PQping since commit f13ea95f9e4 (v10) so the attached patch removes a comment from internal_ping(). -- Euler Taveira EDB https://www.enterprisedb.com/ From 3055de48fe71f47df357114c7a42db05edcdb290 Mon Sep 17 00:00:00

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-01-06 Thread Euler Taveira
is worth. You can use IPC::Run to start psql in background. See examples in src/test/recovery. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2022-01-07 Thread Euler Taveira
On Fri, Jan 7, 2022, at 3:35 AM, Amit Kapila wrote: > On Fri, Jan 7, 2022 at 9:44 AM Amit Kapila wrote: > > > > On Thu, Jan 6, 2022 at 6:42 PM Euler Taveira wrote: > > > > > > IMO we shouldn't reuse ReorderBufferChangeType. For a long-ter

Re: row filtering for logical replication

2022-01-07 Thread Euler Taveira
PubWhereClauses->transformWhereClause->coerce_to_boolean > before storing where clause expression. It is not clear to me why that > is required? We might want to add a comment if that is required. It is redundant. It seems an additional safeguard that we should be removed. Good catch. -

Re: Logging replication state changes

2022-01-08 Thread Euler Taveira
hange the message level but to implement a per-module log_min_messages. This idea is in the TODO [1] for more than a decade. Check the archives. I agree with Tom that the referred messages are noisy, hence, DEBUG1 is fine for it. [1] https://wiki.postgresql.org/wiki/Todo -- Euler Taveira ED

Re: a verbose option for autovacuum

2021-03-17 Thread Euler Taveira
looks good to me. There is just a small issue (double space after 'if') that I fixed in the attached version. -- Euler Taveira EDB https://www.enterprisedb.com/ diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 8341879d89..35be34ea06 100644 ---

Re: cleanup temporary files after crash

2021-03-18 Thread Euler Taveira
; Then, once the second session gets waiting on the tuple, kill the > backend. We might as well test that there actually is a temp file first, > and then test that it disappeared. Your suggestion works for me. Maybe you could use less rows in the session 2 query. I experimented with

Re: cleanup temporary files after crash

2021-03-18 Thread Euler Taveira
and increase the number of > rows, to consistently get temp files. Yeah. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: cleanup temporary files after crash

2021-03-18 Thread Euler Taveira
On Thu, Mar 18, 2021, at 6:00 PM, Euler Taveira wrote: > On Thu, Mar 18, 2021, at 5:51 PM, Tomas Vondra wrote: >> OK. Can you prepare a patch with the proposed test approach? > I'm on it. What do you think about this patch? -- Euler Taveira EDB https://www.enterprisedb.com/

Re: cleanup temporary files after crash

2021-03-18 Thread Euler Taveira
ded - s close ;-) So let's make it 2000. My 32-bit laptop needs some repairs so I blindly chose 1k rows. > We might as well check that the temp file actually exists, before > killing the backend. Just to be sure. Do you mean with remove_temp_files_after_crash = on? New version atta

Re: cleanup temporary files after crash

2021-03-19 Thread Euler Taveira
backend, we could use a pool_query_until() before SIGKILL to wait the temporary file being created. Do you think this modification will make this test more stable? -- Euler Taveira EDB https://www.enterprisedb.com/ diff --git a/src/test/recovery/t/022_crash_temp_files.pl b/src/test/recove

Re: cleanup temporary files after crash

2021-03-19 Thread Euler Taveira
e test than the code itself. :-/ > My brain hurts from reading too much Perl today ... I feel the same when I have to deal with Perl code. It seems the animals are happy with this fix. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-03-21 Thread Euler Taveira
icationcmds.c uses Relation everywhere so I decided to create a new struct to store Relation and qual as a list item. It also minimizes the places you have to modify. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-03-21 Thread Euler Taveira
was leftover from a previous patch set; * add new tests to improve coverage (INSERT/UPDATE/DELETE to exercise the row filter code). -- Euler Taveira EDB https://www.enterprisedb.com/ From a6d893be0091bc8cd8569ac380e6f628263d31c0 Mon Sep 17 00:00:00 2001 From: Euler Taveira Date: Mon, 18 Jan 20

Re: Add docs stub for recovery.conf

2021-03-28 Thread Euler Taveira
patible or Obsolete Features". + trigger_file + promote_trigger_file + +setting has been renamed to + + There should be a period after the . -- Euler Taveira EDB https://www.enterprisedb.com/

Re: row filtering for logical replication

2021-03-28 Thread Euler Taveira
message. At this time, I didn't include the patch that changes the log_min_messages in the row filter regression test. It was part of this patch set for testing purposes only. I don't expect the patch that measures row filter performance to be included but I'm including it again

Re: row filtering for logical replication

2021-03-29 Thread Euler Taveira
er it. It seems somehow your subscriber close the replication connection causing this issue. Can you reproduce it consistently? If so, please share your steps. -- Euler Taveira EDB https://www.enterprisedb.com/ test-row-filter-pgbench.sh Description: application/shellscript

Re: row filtering for logical replication

2021-03-30 Thread Euler Taveira
On Tue, Mar 30, 2021, at 8:23 AM, Amit Kapila wrote: > On Mon, Mar 29, 2021 at 6:47 PM Euler Taveira <mailto:euler%40eulerto.com>> wrote: > > > Few comments: > == > 1. How can we specify row filters for multiple tables for a > publication? Consider a cas

Re: Any objections to implementing LogicalDecodeMessageCB for pgoutput?

2021-04-02 Thread Euler Taveira
plugin. */ Someone that is inspecting the code in the future could possibly check this discussion to understand why this function isn't available. This new patch set version has 2 patches that is because there are 2 separate changes: parse_output_parameters() refactor and logical decoding message

Re: Any objections to implementing LogicalDecodeMessageCB for pgoutput?

2021-04-05 Thread Euler Taveira
since the scope is beyond this feature. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-04-05 Thread Euler Taveira
uitable error message according to the object class (table vs foreign table). While at it, separate unlogged/temp table error message into 2 messages. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: use AV worker items infrastructure for GIN pending list's cleanup

2021-04-05 Thread Euler Taveira
tgresql.org/docs/current/gin-tips.html > To avoid fluctuations in observed response time, it's desirable to have > pending-list cleanup occur in the background (i.e., via autovacuum). Could you provide a link from the previous discussion? -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Any objections to implementing LogicalDecodeMessageCB for pgoutput?

2021-04-07 Thread Euler Taveira
On Wed, Apr 7, 2021, at 2:20 AM, Amit Kapila wrote: > I have pushed this work and updated the CF entry accordingly. Great. Thank you. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Why is specifying oids = false multiple times in create table is silently ignored?

2021-04-07 Thread Euler Taveira
s(see > below)? Shouldn't we throw an error for consistency with other options? > If you look at transformReloptions(), the no-op code is just a hack. Such a patch should add 'oids' as a reloption to test for multiple occurrences. Although, CREATE TABLE says you can use 'oids=false', Storage Parameters section does not mention it as a parameter. The code is fine as is. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: missing documentation for streaming in-progress transactions

2021-04-08 Thread Euler Taveira
ill be same as xid of the transaction for top-level +transactions). + Although, sub-transaction is also used in the documentation, I suggest to use subtransaction. Maybe change the other sub-transaction occurrences too. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: deferred primary key and logical replication

2020-07-27 Thread Euler Taveira
status of this patch is: Waiting on Author > Thanks for testing. I attached a rebased patch. -- Euler Taveira http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From c1ad1581962a56c83183bec1501df6f54406db89 Mon Sep 17 00:

Re: emit recovery stats via a new file or a new hook

2021-11-10 Thread Euler Taveira
implement. It seems more effort than it's worth. +1 for storing this data into the stats system. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-11-10 Thread Euler Taveira
is fine to add additional DEBUG messages if there isn't a similar one yet. If at least the message level were module-controlled, you could modify a setting to gather more messages from a specific module. Unfortunately, that's not possible so we should avoid superfluous messages. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-11-11 Thread Euler Taveira
in your own script/program. I like Alvaro's idea of implementing a ring buffer for this kind of activity. This implementation could be applied to checkpoints, archiving, recovery, replication, base backup, WAL usage, connections and others. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

2021-11-11 Thread Euler Taveira
lude a DELETE command, wait the initial table sync to avoid failing a subsequent test and improve some comments. -- Euler Taveira EDB https://www.enterprisedb.com/ From e81e0bb6c6796e7fd692b7d6643db8fb6770d9a1 Mon Sep 17 00:00:00 2001 From: "houzj.fnst" Date: Thu, 11 Nov 2021 18:21:

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-12 Thread Euler Taveira
stence as a 2nd parameter and move those messages to it? I experiment this idea with the attached patch. The idea is to provide a unique function that reports accurate detail messages. -- Euler Taveira EDB https://www.enterprisedb.com/ From f068d4688a95c8e8c5a98d8d6f1847ddfafda43c Mon Sep 1

Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

2021-11-12 Thread Euler Taveira
at is the case I wonder why not simply register > > invalidation without any check in the for loop as was the case with > > Tang's original patch? > > OK, I also think the code in Tang's original patch is fine. > Attach the patch which register invalidation without any check in the for > loop. WFM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-13 Thread Euler Taveira
On Sat, Nov 13, 2021, at 12:00 AM, Bharath Rupireddy wrote: > On Sat, Nov 13, 2021 at 12:06 AM Euler Taveira wrote: > > > Here's a rebased v8 patch. Please review it. > > > > This improves the user experience by increasing the granularity of error > > reportin

  1   2   3   4   5   >