Re: Supply restore_command to pg_rewind via CLI argument

2022-03-22 Thread Alexey Kondratov
Hi, On Tue, Mar 22, 2022 at 3:32 AM Andres Freund wrote: > > Doesn't apply once more: http://cfbot.cputube.org/patch_37_3213.log > Thanks for the reminder, a rebased version is attached. Regards -- Alexey Kondratov From df56b5c7b882e781fdc0b92e7a83331f0baab094 Mon Sep 17 00:0

Re: Supply restore_command to pg_rewind via CLI argument

2021-08-27 Thread Alexey Kondratov
tgresql.conf anyway. > > Besides this patch looks good and is ready for committer IMV. > -- Alexey Kondratov v2-0001-Allow-providing-restore_command-as-a-command-line.patch Description: Binary data

Re: Supply restore_command to pg_rewind via CLI argument

2021-06-29 Thread Alexey Kondratov
On Fri, Jun 18, 2021 at 10:06 PM Alexey Kondratov wrote: > On Fri, Jun 18, 2021 at 5:42 PM Andrey Borodin wrote: > > > > If we run 'pg_rewind --restore-target-wal' there must be restore_command in > > config of target installation. But if the config is not with

Re: Supply restore_command to pg_rewind via CLI argument

2021-06-18 Thread Alexey Kondratov
will have a look, maybe I even already have this patch separately. I remember that we were considering adding this option to PostgresPro, when we did a backport of this feature. -- Alexey Kondratov

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-20 Thread Alexey Kondratov
ss. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm index db47a97d196..f7b488ed464 100644 --- a/src/test/perl/PostgresNode.pm +++ b/src/test/perl/PostgresNode.pm @@ -119

Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-19 Thread Alexey Kondratov
t required TCP for stable running. That way, if it really could happen why not to just skip binding to 127.0.0/24 addresses other than 127.0.0.1 outside of Linux/Windows as per attached patch_PostgresNode.diff? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com R

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-03 Thread Alexey Kondratov
On 2021-02-03 09:37, Michael Paquier wrote: On Tue, Feb 02, 2021 at 10:32:19AM +0900, Michael Paquier wrote: On Mon, Feb 01, 2021 at 06:28:57PM +0300, Alexey Kondratov wrote: > Hm, IIUC, REINDEX CONCURRENTLY doesn't use either of them. It directly uses > index_create() wi

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-01 Thread Alexey Kondratov
On 2021-01-30 05:23, Michael Paquier wrote: On Fri, Jan 29, 2021 at 08:56:47PM +0300, Alexey Kondratov wrote: On 2021-01-28 14:42, Alexey Kondratov wrote: No, you are right, we are doing REINDEX with AccessExclusiveLock as it was before. This part is a more specific one. It only applies to

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-29 Thread Alexey Kondratov
On 2021-01-28 14:42, Alexey Kondratov wrote: On 2021-01-28 00:36, Alvaro Herrera wrote: I didn't look at the patch closely enough to understand why you're trying to do something like CLUSTER, VACUUM FULL or REINDEX without holding full AccessExclusiveLock on the relation. But do ke

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-28 Thread Alexey Kondratov
On 2021-01-28 00:36, Alvaro Herrera wrote: On 2021-Jan-28, Alexey Kondratov wrote: I have read more about lock levels and ShareLock should prevent any kind of physical modification of indexes. We already hold ShareLock doing find_all_inheritors(), which is higher than ShareUpdateExclusiveLock

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-27 Thread Alexey Kondratov
On 2021-01-27 06:14, Michael Paquier wrote: On Wed, Jan 27, 2021 at 01:00:50AM +0300, Alexey Kondratov wrote: In the new 0002 I moved ACL check to the upper level, i.e. ExecReindex(), and removed expensive text generation in test. Not touched yet some of your previously raised concerns. Also

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-26 Thread Alexey Kondratov
On 2021-01-26 09:58, Michael Paquier wrote: On Mon, Jan 25, 2021 at 11:11:38PM +0300, Alexey Kondratov wrote: I updated comment with CCI info, did pgindent run and renamed new function to SetRelationTableSpace(). New patch is attached. [...] Yeah, all these checks we complicated from the

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-25 Thread Alexey Kondratov
On 2021-01-25 11:07, Michael Paquier wrote: On Fri, Jan 22, 2021 at 05:07:02PM +0300, Alexey Kondratov wrote: I have updated patches accordingly and also simplified tablespaceOid checks and assignment in the newly added SetRelTableSpace(). Result is attached as two separate patches for an ease

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-22 Thread Alexey Kondratov
On 2021-01-22 00:26, Justin Pryzby wrote: On Thu, Jan 21, 2021 at 11:48:08PM +0300, Alexey Kondratov wrote: Attached is a new patch set of first two patches, that should resolve all the issues raised before (ACL, docs, tests) excepting TOAST. Double thanks for suggestion to add more tests with

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Alexey Kondratov
On 2021-01-21 17:06, Alexey Kondratov wrote: On 2021-01-21 04:41, Michael Paquier wrote: There are no tests for partitioned tables, aka we'd want to make sure that the new partitioned index is on the correct tablespace, as well as all its leaves. It may be better to have at least two l

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Alexey Kondratov
On 2021-01-21 04:41, Michael Paquier wrote: On Wed, Jan 20, 2021 at 03:34:39PM -0300, Alvaro Herrera wrote: On 2021-Jan-20, Alexey Kondratov wrote: Ugh, forgot to attach the patches. Here they are. Yeah, looks reasonable. + + if (changed) + /* Record dependency on

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alexey Kondratov
On 2021-01-20 21:08, Alexey Kondratov wrote: On 2021-01-20 18:54, Alvaro Herrera wrote: On 2021-Jan-20, Alvaro Herrera wrote: On 2021-Jan-20, Michael Paquier wrote: > +/* > + * This is mostly duplicating ATExecSetTableSpaceNoStorage, > + * which should maybe be factored out to

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alexey Kondratov
rns regarding reindex of partitions. I am going to look closer on it tomorrow. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-13 Thread Alexey Kondratov
is clever enough to link your email to the same thread even with different subject. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index 9904a76387..43cfdeaa6b 100644 ---

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-12-30 Thread Alexey Kondratov
On 2020-12-30 17:59, Bharath Rupireddy wrote: On Wed, Dec 30, 2020 at 5:20 PM Alexey Kondratov wrote: On 2020-12-30 09:10, Bharath Rupireddy wrote: I still have some doubts that it is worth of allowing to call postgres_fdw_disconnect() in the explicit transaction block, since it adds a lot of

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-12-30 Thread Alexey Kondratov
his will be corrected later by pg_indent, but still. In this comment section following points 1) and 2) have a different combination of tabs/spaces. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyFrom 82b82b901e8f0ca84e1b21454a3b68f4fd8f0016

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-23 Thread Alexey Kondratov
stent: * we have ReindexOptions, but VacuumParams * and ReindexOptions->flags, but VacuumParams->options And the last one, you have used bits32 for Cluster/ReindexOptions, but left VacuumParams->options as int. Maybe we should also change it to bits32 for consistency? Regards -- Al

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-23 Thread Alexey Kondratov
ms to be redundant. For some reason I have mixed these refactorings separated by a dozen of versions... Thanks -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-12-17 Thread Alexey Kondratov
t allowing to call a non-transactional by its nature function in the transaction block. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-15 Thread Alexey Kondratov
g further since I've done too much rebasing with these ideas going back and forth and back. Yes, we have moved a bit from my original patch set in the thread with all this refactoring. However, all the consequent patches are heavily depend on this interface, so let us decide first on the proposed refactoring. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-04 Thread Alexey Kondratov
agine a lot of users trying to constantly move indexes to the global tablespace, so it is also OK to put this check deeper into guts. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/backend/commands/indexcmds.c b/src/backend/comma

Re: Notes on physical replica failover with logical publisher or subscriber

2020-11-30 Thread Alexey Kondratov
ith physical replication --- ex-primary may accept some xacts after promotion of new primary, so their history diverges and old primary should be rewound before being returned as standby (subscriber). Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-30 Thread Alexey Kondratov
p of 873ea9ee to use 'utility_option_list' instead of 'common_option_list'. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyFrom ac3b77aec26a40016784ada9dab8b9059f424fa4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 31

Re: Printing LSN made easy

2020-11-27 Thread Alexey Kondratov
charbuf[MAXPG_LSNLEN + 1]; + + snprintf(buf, sizeof(buf), LSN_FORMAT, LSN_FORMAT_ARG(lsn)); + + return pstrdup(buf); +} Would it be a bit more straightforward if we palloc buf initially and just return a pointer instead of doing pstrdup()? Regards -- Alexey K

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-25 Thread Alexey Kondratov
es along with remote connections? I mean, that it would be a nice to have hook from the extensibility perspective, but postgres_fdw_disconnect() still makes sense, since it does a very narrow and specific job. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-24 Thread Alexey Kondratov
On 2020-11-24 06:52, Bharath Rupireddy wrote: Thanks for the review comments. On Mon, Nov 23, 2020 at 9:57 PM Alexey Kondratov wrote: > v1-0001-postgres_fdw-function-to-discard-cached-connections.patch This patch looks pretty straightforward for me, but there are some things to be addres

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-23 Thread Alexey Kondratov
ince we set entry->xact_depth to zero at the end of xact, then it was used if it is not zero. It is set to 1 by begin_remote_xact() called by GetConnection(), so everything seems to be fine. Otherwise, both patches seem to be working as expected. I am going to have a look on the last two patches a bit later. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-19 Thread Alexey Kondratov
On 2020-11-19 07:11, Bharath Rupireddy wrote: On Wed, Nov 18, 2020 at 10:32 PM Alexey Kondratov wrote: Thanks! I will make separate patches and post them soon. >> Attached is a small POC patch, which implements this contrib-level >> postgres_fdw.keep_connections GUC. What

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-18 Thread Alexey Kondratov
On 2020-11-18 16:39, Bharath Rupireddy wrote: Thanks for the interest shown! On Wed, Nov 18, 2020 at 1:07 AM Alexey Kondratov wrote: Regarding the initial issue I prefer point #3, i.e. foreign server option. It has a couple of benefits IMO: 1) it may be set separately on per foreign server

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-17 Thread Alexey Kondratov
option by setting this GUC on per session basis. Attached is a small POC patch, which implements this contrib-level postgres_fdw.keep_connections GUC. What do you think? [1] https://www.postgresql.org/message-id/CALj2ACUFNydy0uo0JL9A1isHQ9pFe1Fgqa_HVanfG6F8g21nSQ%40mail.gmail.com Regards

Re: Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-12 Thread Alexey Kondratov
On 2020-11-11 06:59, Tom Lane wrote: Alexey Kondratov writes: After looking on the autoprewarm code more closely I have realised that this 'double dump' issues was not an issues at all. I have just misplaced a debug elog(), so its second output in the log was only indicati

Re: Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-10 Thread Alexey Kondratov
On 2020-11-09 23:25, Tom Lane wrote: Alexey Kondratov writes: On 2020-11-09 21:53, Tom Lane wrote: 0002 seems like a pretty clear bug fix, though I wonder if this is exactly what we want to do going forward. It seems like a very large fraction of the callers of TimestampDifference would

Re: Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-09 Thread Alexey Kondratov
On 2020-11-09 21:53, Tom Lane wrote: Alexey Kondratov writes: After fixing this issue I have noticed that it still dumps blocks twice at each timeout (here I set autoprewarm_interval to 15s): ... This happens because at timeout time we were using continue, but actually we still have to wait

Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-09 Thread Alexey Kondratov
o you think? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyFrom 6d4bab7f21c3661dd4dd5a0de7e097b1de3f642c Mon Sep 17 00:00:00 2001 From: Alexey Kondratov Date: Mon, 9 Nov 2020 19:24:55 +0300 Subject: [PATCH v1 3/3] pg_prewarm: refactor autopr

Re: Global snapshots

2020-09-21 Thread Alexey Kondratov
On 2020-09-18 00:54, Bruce Momjian wrote: On Tue, Sep 8, 2020 at 01:36:16PM +0300, Alexey Kondratov wrote: Thank you for the link! After a quick look on the Sawada-san's patch set I think that there are two major differences: 1. There is a built-in foreign xacts resolver in the [1],

Re: Concurrency issue in pg_rewind

2020-09-17 Thread Alexey Kondratov
On 2020-09-17 15:27, Alexander Kukushkin wrote: On Thu, 17 Sep 2020 at 14:04, Alexey Kondratov wrote: With --restore-target-wal pg_rewind is trying to call restore_command on its own and it can happen at two stages: 1) When pg_rewind is trying to find the last checkpoint preceding a

Re: Concurrency issue in pg_rewind

2020-09-17 Thread Alexey Kondratov
gh keeping arbitrary files inside PGDATA does not look like a good idea for me, I do not see anything criminal in skipping non-existing file, when executing a file map by pg_rewind. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Global snapshots

2020-09-10 Thread Alexey Kondratov
On 2020-09-09 20:29, Fujii Masao wrote: On 2020/09/09 2:00, Alexey Kondratov wrote: According to the Sawada-san's v25 0002 the logic is pretty much the same there: +2. Pre-Commit phase (1st phase of two-phase commit) +3. Commit locally +Once we've prepared all of them,

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-09-09 Thread Alexey Kondratov
es parenthesized option list. All the checks and validations are performed at the corresponding command code. This analyze_keyword is actually doing only an ANALYZE word normalization if it's used as an option. Why it could be harmful? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Global snapshots

2020-09-09 Thread Alexey Kondratov
On 2020-09-09 08:35, Masahiko Sawada wrote: On Wed, 9 Sep 2020 at 02:00, Alexey Kondratov wrote: On 2020-09-08 14:48, Fujii Masao wrote: > > IIUC, yes, the information required for automatic resolution is > WAL-logged and the standby tries to resolve those orphan transactions > fr

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-09 Thread Alexey Kondratov
On 2020-09-09 11:45, Andrey V. Lepikhov wrote: On 9/8/20 8:34 PM, Alexey Kondratov wrote: On 2020-09-08 17:00, Amit Langote wrote: wrote: On 2020-09-08 10:34, Amit Langote wrote: Another ambiguous part of the refactoring was in changing InitResultRelInfo() arguments: @@ -1278,6 +1280,7

Re: Global snapshots

2020-09-08 Thread Alexey Kondratov
On 2020-09-08 14:48, Fujii Masao wrote: On 2020/09/08 19:36, Alexey Kondratov wrote: On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-08 Thread Alexey Kondratov
On 2020-09-08 17:00, Amit Langote wrote: Hi Alexey, On Tue, Sep 8, 2020 at 6:29 PM Alexey Kondratov wrote: On 2020-09-08 10:34, Amit Langote wrote: > Any thoughts on the taking out the refactoring changes out of the main > patch as I suggested? > +1 for splitting the patch. It w

Re: Global snapshots

2020-09-08 Thread Alexey Kondratov
On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it solves these errors handling issues above and tries to add proper comments

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-08 Thread Alexey Kondratov
this check doesn't look costly. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Global snapshots

2020-09-04 Thread Alexey Kondratov
plements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it solves these errors handling issues above and tries to add proper comments everywhere. I think, that 0003 should be rebased on the top of it, or it could be a first patch in the set, since it may

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-09-01 Thread Alexey Kondratov
me how exactly. That way, I'd be glad if Michael could reword his explanation, so it'd more clear for me as well. BTW, I've started doing a review of the last patch set yesterday and will try to post some comments later. Regards -- Alexey Kondratov Postgres Profession

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-07-28 Thread Alexey Kondratov
On 2020-07-28 03:33, Andrey Lepikhov wrote: 27.07.2020 21:34, Alexey Kondratov пишет: Hi Andrey, On 2020-07-23 09:23, Andrey V. Lepikhov wrote: On 7/16/20 2:14 PM, Amit Langote wrote: Amit Langote EnterpriseDB: http://www.enterprisedb.com Version 5 of the patch. With changes caused by

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-07-27 Thread Alexey Kondratov
patched Postgres binaries should be available in the PATH). It works well with master and fails with your patch applied. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company#!/usr/bin/env sh pg_ctl -D node1 stop > /dev/null pg_ctl -D node2

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Alexey Kondratov
On 2020-07-16 19:35, Etsuro Fujita wrote: On Thu, Jul 16, 2020 at 8:56 PM Andrey Lepikhov wrote: On 7/16/20 9:55 AM, Etsuro Fujita wrote: >>>> On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov >>>> wrote: >>>>> Some real-life test queries sh

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Alexey Kondratov
On 2020-07-16 14:56, Andrey Lepikhov wrote: On 7/16/20 9:55 AM, Etsuro Fujita wrote: On Wed, Jul 15, 2020 at 9:02 PM Etsuro Fujita wrote: On Wed, Jul 15, 2020 at 12:12 AM Alexey Kondratov wrote: On 2020-07-14 15:27, Ashutosh Bapat wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-14 Thread Alexey Kondratov
On 2020-07-14 15:27, Ashutosh Bapat wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wrote: I built a simple two node multi-tenant schema for tests, which can be easily set up with attached scripts. It creates three tables (companies, users, documents) distributed over two nodes

Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-13 Thread Alexey Kondratov
lobus/8fba33241f68be2e3765d27bf04882a3 [3] https://www.postgresql.org/message-id/flat/CAFT%2BaqL1Tt0qfYqjHH%2BshwPoW8qdFjpJ8vBR5ABoXJDUcHyN1w%40mail.gmail.com Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyDROP TABLE IF EXISTS companies

Re: [PATCH] Allow to specify restart_lsn in pg_create_physical_replication_slot()

2020-06-26 Thread Alexey Kondratov
On 2020-06-23 04:18, Michael Paquier wrote: On Mon, Jun 22, 2020 at 08:18:58PM +0300, Alexey Kondratov wrote: Things get worse when we allow specifying an older LSN, since it has a higher chances to be at the horizon of deletion by checkpointer. Anyway, if I get it correctly, with a current

Re: [PATCH] Allow to specify restart_lsn in pg_create_physical_replication_slot()

2020-06-22 Thread Alexey Kondratov
On 2020-06-19 21:57, Fujii Masao wrote: On 2020/06/19 23:20, Alexey Kondratov wrote: On 2020-06-19 03:59, Michael Paquier wrote: On Thu, Jun 18, 2020 at 03:39:09PM +0300, Vyacheslav Makarov wrote: If the WAL segment for the specified restart_lsn (STOP_LSN of the backup) exists, then the

Re: [PATCH] Allow to specify restart_lsn in pg_create_physical_replication_slot()

2020-06-19 Thread Alexey Kondratov
ation. [1] https://www.postgresql.org/message-id/flat/20180626071305.GH31353%40paquier.xyz Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Physical replication slot advance is not persistent

2020-06-18 Thread Alexey Kondratov
On 2020-06-16 10:27, Michael Paquier wrote: On Wed, Jun 10, 2020 at 08:57:17PM +0300, Alexey Kondratov wrote: New test reproduces this issue well. Left it running for a couple of hours in repeat and it seems to be stable. Thanks for testing. I have been thinking about the minimum xmin and

Re: Physical replication slot advance is not persistent

2020-06-10 Thread Alexey Kondratov
uot; ); chomp($phys_restart_lsn_pre); we can safely use $current_lsn used for pg_replication_slot_advance(), since reatart_lsn is set as is there. It may make the test a bit simpler as well. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Physical replication slot advance is not persistent

2020-06-09 Thread Alexey Kondratov
ole machinery of WAL holding and trimming is worth to be tested more thoroughly. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 1b929a603e..8e543e2

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-06-08 Thread Alexey Kondratov
esql.org/message-id/784fa7dc-414b-9dc9-daae-138033db298c%40postgrespro.ru Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-10 Thread Alexey Kondratov
On 2020-04-10 05:25, Fujii Masao wrote: On 2020/04/10 3:16, Alexey Kondratov wrote: Just another idea in case if one will still decide to go with a separate statement + BEGIN integration instead of a function. We could use parenthesized options list here. This is already implemented for

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-09 Thread Alexey Kondratov
2-a9b2649cdd1f%402ndquadrant.com [2] https://www.postgresql.org/message-id/20200401060334.GB142683%40paquier.xyz Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-07 Thread Alexey Kondratov
On 2020-04-06 21:44, Justin Pryzby wrote: On Mon, Apr 06, 2020 at 08:43:46PM +0300, Alexey Kondratov wrote: +/* XXX: reusing reindex_option_list */ + | CLUSTER opt_verbose '(' reindex_option_list ')' qualified_name cluster_index_specification Coul

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-06 Thread Alexey Kondratov
d idea. Yes, it saves us a few checks here and there now, but it may reduce a further reusability of these internal routines in the future. XXX: for cluster/vacuum, it might be more friendly to check before clustering the table, rather than after clustering and re-indexing.

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-03 Thread Alexey Kondratov
It seems to be equivalent to pg_sleep, doesn't it? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-04-01 Thread Alexey Kondratov
for doing that! And to all others who participated. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-31 Thread Alexey Kondratov
On 2020-03-30 21:34, Justin Pryzby wrote: On Mon, Mar 30, 2020 at 09:02:22PM +0300, Alexey Kondratov wrote: Hmm, I went through the well known to me SQL commands in Postgres and a bit more. Parenthesized options list is mostly used in two common cases: There's also ANALYZE(VERBOSE), RE

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-30 Thread Alexey Kondratov
ng with syntax as it is in 0001-0002 2) Implement tail syntax for VACUUM, but with limitation for VACUUM FULL of the entire database + TABLESPACE change 3) Change TABLESPACE to a fully reserved word Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-30 Thread Alexey Kondratov
Let us use camel case 'restoreCommand' here as in the header for tidiness. I have left 0001 intact, but fixed all these small remarks in the 0002. Please, find it attached. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company From f

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-26 Thread Alexey Kondratov
have tried to do what you proposed and attached is a small patch, that introduces xlogarchive.h. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/backend/access/transam/timeline.c b/src/backend/access/transam/timeline.c ind

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Alexey Kondratov
to rewrite VACUUM's part again with OptTableSpace. Maybe I will manage it this time. I will take into account all your text edits as well. Thanks -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Alexey Kondratov
On 2020-03-26 02:40, Justin Pryzby wrote: On Thu, Mar 12, 2020 at 08:08:46PM +0300, Alexey Kondratov wrote: On 09.03.2020 23:04, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 08:53:04AM -0600, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 03:35:27PM +0300, Alexey Kondratov wrote: tests for that

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-12 Thread Alexey Kondratov
Hi Justin, On 09.03.2020 23:04, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 08:53:04AM -0600, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 03:35:27PM +0300, Alexey Kondratov wrote: Anyway, new version is attached. It is rebased in order to resolve conflicts with a recent fix of REINDEX

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-12 Thread Alexey Kondratov
which is frontend-only. Is it there for historical reasons only or something else? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Conflict handling for COPY FROM

2020-03-10 Thread Alexey Kondratov
works just fine. [1] https://github.com/postgres/postgres/blob/0a42a2e9/src/backend/tcop/postgres.c#L1178 [2] https://github.com/postgres/postgres/blob/0a42a2e9/src/backend/tcop/postgres.c#L1226 Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-05 Thread Alexey Kondratov
On 05.03.2020 09:24, Michael Paquier wrote: On Wed, Mar 04, 2020 at 08:14:20PM +0300, Alexey Kondratov wrote: - I did not actually get why you don't check for a missing command when using wait_result_is_any_signal. In this case I'd think that it is better to exit immediately as follo

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-04 Thread Alexey Kondratov
On 04.03.2020 10:45, Michael Paquier wrote: On Mon, Mar 02, 2020 at 08:59:49PM +0300, Alexey Kondratov wrote: All other remarks look clear for me, so I fix them in the next patch version, thanks. Already done as per the attached, with a new routine named getRestoreCommand() and more done

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-02 Thread Alexey Kondratov
r for me, so I fix them in the next patch version, thanks. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com The Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-02-29 Thread Alexey Kondratov
REINDEX CONCURRENTLY + temp relations, and includes this small comment fix. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com The Russian Postgres Company From d2b7a5fa2e11601759b47af0c142a7824ef907a2 Mon Sep 17 00:00:00 2001 From: Alexey Kondratov Date: Mon, 30 Dec

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-02-28 Thread Alexey Kondratov
On 2020-02-28 09:43, Michael Paquier wrote: On Thu, Feb 27, 2020 at 06:29:34PM +0300, Alexey Kondratov wrote: On 2020-02-27 16:41, Alexey Kondratov wrote: > > New version of the patch is attached. Thanks again for your review. > Last patch (v18) got a conflict with one of toda

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-02-27 Thread Alexey Kondratov
On 2020-02-27 16:41, Alexey Kondratov wrote: New version of the patch is attached. Thanks again for your review. Last patch (v18) got a conflict with one of today commits (05d8449e73). Rebased version is attached. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com The

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-02-27 Thread Alexey Kondratov
OK, shorter is better. I think that it would be saner to check the return status of ConstructRestoreCommand() in xlogarchive.c as a sanity check, with an elog(ERROR) if not 0, as that should never happen. Added. New version of the patch is attached. Thanks again f

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-02-26 Thread Alexey Kondratov
corrections above. It is made with default git format-patch format, but yours were in a slightly different format, so I only was able to apply them with git am --patch-format=stgit. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com The Russian Postgres Company From fa2fc

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-01-31 Thread Alexey Kondratov
much better for a future support. New version is attached. Do you have any other comments or objections? Regards -- Alexey From 7af0b3642f6218c764eee361e013f24bfb43ffbe Mon Sep 17 00:00:00 2001 From: Alexey Kondratov Date: Fri, 31 Jan 2020 20:08:13 +0300 Subject: [PATCH v14] pg_rewind: Add opti

Re: Physical replication slot advance is not persistent

2020-01-31 Thread Alexey Kondratov
On 30.01.2020 05:19, Michael Paquier wrote: On Wed, Jan 29, 2020 at 05:10:20PM +0900, Kyotaro Horiguchi wrote: Looks perfect. Thanks Horiguchi-san and others. Applied and back-patched down to 11. Great! Thanks for getting this done. -- Alexey Kondratov Postgres Professional https

Re: Physical replication slot advance is not persistent

2020-01-28 Thread Alexey Kondratov
e, the term 'schedule' is used for VACUUM, constraint check or checkpoint itself. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Physical replication slot advance is not persistent

2020-01-16 Thread Alexey Kondratov
On 09.01.2020 09:36, Kyotaro Horiguchi wrote: Hello. At Sun, 29 Dec 2019 15:12:16 +0300, Alexey Kondratov wrote in On 2019-12-26 16:35, Alexey Kondratov wrote: Another concern is that ReplicationSlotIsDirty is added with the only one user. It also cannot be used by SaveSlotToPath due to the

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-01-04 Thread Alexey Kondratov
On 2019-12-02 11:21, Michael Paquier wrote: On Wed, Nov 27, 2019 at 08:47:06PM +0300, Alexey Kondratov wrote: Thus, I cannot get your point correctly here. Can you, please, elaborate a little bit more your concerns? The case of REINDEX CONCURRENTLY is pretty simple, because a new relation

Re: Physical replication slot advance is not persistent

2019-12-29 Thread Alexey Kondratov
On 2019-12-26 16:35, Alexey Kondratov wrote: Another concern is that ReplicationSlotIsDirty is added with the only one user. It also cannot be used by SaveSlotToPath due to the simultaneous usage of both flags dirty and just_dirtied there. In that way, I hope that we should call

Re: Physical replication slot advance is not persistent

2019-12-26 Thread Alexey Kondratov
On 26.12.2019 11:33, Kyotaro Horiguchi wrote: At Wed, 25 Dec 2019 20:28:04 +0300, Alexey Kondratov wrote in Yep, it helps with physical replication slot persistence after advance, but the whole validation (moveto <= endlsn) does not make sense for me. The value of moveto should be >

Re: Physical replication slot advance is not persistent

2019-12-25 Thread Alexey Kondratov
On 25.12.2019 16:51, Alexey Kondratov wrote: On 25.12.2019 07:03, Kyotaro Horiguchi wrote: As the result I think what is needed there is just checking if the returned lsn is equal or larger than moveto. Doen't the following change work? -    if (XLogRecPtrIsInvalid(endlsn)) +    if (m

Re: Physical replication slot advance is not persistent

2019-12-25 Thread Alexey Kondratov
On 25.12.2019 07:03, Kyotaro Horiguchi wrote: At Tue, 24 Dec 2019 20:12:32 +0300, Alexey Kondratov wrote in I dig into the code and it happens because of this if statement:     /* Update the on disk state when lsn was updated. */     if (XLogRecPtrIsInvalid(endlsn

Physical replication slot advance is not persistent

2019-12-24 Thread Alexey Kondratov
ogRecPtrIsInvalid(endlsn))' and now pg_logical_replication_slot_advance and pg_physical_replication_slot_advance return InvalidXLogRecPtr if no-op. What do you think? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company P.S. CCed Simon and Michael as

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-20 Thread Alexey Kondratov
, but still. We already have hundreds of GUCs and it is easy for a user to build a sub-optimal configuration, so does this overprotection make sense? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-12-03 Thread Alexey Kondratov
On 01.12.2019 5:57, Michael Paquier wrote: On Thu, Sep 26, 2019 at 03:08:22PM +0300, Alexey Kondratov wrote: As Alvaro correctly pointed in the nearby thread [1], we've got an interference regarding -R command line argument. I agree that it's a good idea to reserve -R for recovery con

  1   2   >