Re: Supply restore_command to pg_rewind via CLI argument

2022-03-22 Thread Alexey Kondratov
Hi, On Tue, Mar 22, 2022 at 3:32 AM Andres Freund wrote: > > Doesn't apply once more: http://cfbot.cputube.org/patch_37_3213.log > Thanks for the reminder, a rebased version is attached. Regards -- Alexey Kondratov From df56b5c7b882e781fdc0b92e7a83331f0baab094 Mon Sep 17 00:0

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-04 Thread Alexey Kondratov
agine a lot of users trying to constantly move indexes to the global tablespace, so it is also OK to put this check deeper into guts. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/backend/commands/indexcmds.c b/src/backend/comma

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-15 Thread Alexey Kondratov
g further since I've done too much rebasing with these ideas going back and forth and back. Yes, we have moved a bit from my original patch set in the thread with all this refactoring. However, all the consequent patches are heavily depend on this interface, so let us decide first on the proposed refactoring. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-12-17 Thread Alexey Kondratov
t allowing to call a non-transactional by its nature function in the transaction block. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-23 Thread Alexey Kondratov
ms to be redundant. For some reason I have mixed these refactorings separated by a dozen of versions... Thanks -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-23 Thread Alexey Kondratov
stent: * we have ReindexOptions, but VacuumParams * and ReindexOptions->flags, but VacuumParams->options And the last one, you have used bits32 for Cluster/ReindexOptions, but left VacuumParams->options as int. Maybe we should also change it to bits32 for consistency? Regards -- Al

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-12-30 Thread Alexey Kondratov
his will be corrected later by pg_indent, but still. In this comment section following points 1) and 2) have a different combination of tabs/spaces. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyFrom 82b82b901e8f0ca84e1b21454a3b68f4fd8f0016

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-12-30 Thread Alexey Kondratov
On 2020-12-30 17:59, Bharath Rupireddy wrote: On Wed, Dec 30, 2020 at 5:20 PM Alexey Kondratov wrote: On 2020-12-30 09:10, Bharath Rupireddy wrote: I still have some doubts that it is worth of allowing to call postgres_fdw_disconnect() in the explicit transaction block, since it adds a lot of

Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-09 Thread Alexey Kondratov
o you think? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyFrom 6d4bab7f21c3661dd4dd5a0de7e097b1de3f642c Mon Sep 17 00:00:00 2001 From: Alexey Kondratov Date: Mon, 9 Nov 2020 19:24:55 +0300 Subject: [PATCH v1 3/3] pg_prewarm: refactor autopr

Re: Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-09 Thread Alexey Kondratov
On 2020-11-09 21:53, Tom Lane wrote: Alexey Kondratov writes: After fixing this issue I have noticed that it still dumps blocks twice at each timeout (here I set autoprewarm_interval to 15s): ... This happens because at timeout time we were using continue, but actually we still have to wait

Re: Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-10 Thread Alexey Kondratov
On 2020-11-09 23:25, Tom Lane wrote: Alexey Kondratov writes: On 2020-11-09 21:53, Tom Lane wrote: 0002 seems like a pretty clear bug fix, though I wonder if this is exactly what we want to do going forward. It seems like a very large fraction of the callers of TimestampDifference would

Re: Misuse of TimestampDifference() in the autoprewarm feature of pg_prewarm

2020-11-12 Thread Alexey Kondratov
On 2020-11-11 06:59, Tom Lane wrote: Alexey Kondratov writes: After looking on the autoprewarm code more closely I have realised that this 'double dump' issues was not an issues at all. I have just misplaced a debug elog(), so its second output in the log was only indicati

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-17 Thread Alexey Kondratov
option by setting this GUC on per session basis. Attached is a small POC patch, which implements this contrib-level postgres_fdw.keep_connections GUC. What do you think? [1] https://www.postgresql.org/message-id/CALj2ACUFNydy0uo0JL9A1isHQ9pFe1Fgqa_HVanfG6F8g21nSQ%40mail.gmail.com Regards

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-18 Thread Alexey Kondratov
On 2020-11-18 16:39, Bharath Rupireddy wrote: Thanks for the interest shown! On Wed, Nov 18, 2020 at 1:07 AM Alexey Kondratov wrote: Regarding the initial issue I prefer point #3, i.e. foreign server option. It has a couple of benefits IMO: 1) it may be set separately on per foreign server

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-19 Thread Alexey Kondratov
On 2020-11-19 07:11, Bharath Rupireddy wrote: On Wed, Nov 18, 2020 at 10:32 PM Alexey Kondratov wrote: Thanks! I will make separate patches and post them soon. >> Attached is a small POC patch, which implements this contrib-level >> postgres_fdw.keep_connections GUC. What

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-23 Thread Alexey Kondratov
ince we set entry->xact_depth to zero at the end of xact, then it was used if it is not zero. It is set to 1 by begin_remote_xact() called by GetConnection(), so everything seems to be fine. Otherwise, both patches seem to be working as expected. I am going to have a look on the last two patches a bit later. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-24 Thread Alexey Kondratov
On 2020-11-24 06:52, Bharath Rupireddy wrote: Thanks for the review comments. On Mon, Nov 23, 2020 at 9:57 PM Alexey Kondratov wrote: > v1-0001-postgres_fdw-function-to-discard-cached-connections.patch This patch looks pretty straightforward for me, but there are some things to be addres

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-25 Thread Alexey Kondratov
es along with remote connections? I mean, that it would be a nice to have hook from the extensibility perspective, but postgres_fdw_disconnect() still makes sense, since it does a very narrow and specific job. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Printing LSN made easy

2020-11-27 Thread Alexey Kondratov
charbuf[MAXPG_LSNLEN + 1]; + + snprintf(buf, sizeof(buf), LSN_FORMAT, LSN_FORMAT_ARG(lsn)); + + return pstrdup(buf); +} Would it be a bit more straightforward if we palloc buf initially and just return a pointer instead of doing pstrdup()? Regards -- Alexey K

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-30 Thread Alexey Kondratov
p of 873ea9ee to use 'utility_option_list' instead of 'common_option_list'. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres CompanyFrom ac3b77aec26a40016784ada9dab8b9059f424fa4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 31

Re: Notes on physical replica failover with logical publisher or subscriber

2020-11-30 Thread Alexey Kondratov
ith physical replication --- ex-primary may accept some xacts after promotion of new primary, so their history diverges and old primary should be rewound before being returned as standby (subscriber). Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-13 Thread Alexey Kondratov
is clever enough to link your email to the same thread even with different subject. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index 9904a76387..43cfdeaa6b 100644 ---

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alexey Kondratov
rns regarding reindex of partitions. I am going to look closer on it tomorrow. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alexey Kondratov
On 2021-01-20 21:08, Alexey Kondratov wrote: On 2021-01-20 18:54, Alvaro Herrera wrote: On 2021-Jan-20, Alvaro Herrera wrote: On 2021-Jan-20, Michael Paquier wrote: > +/* > + * This is mostly duplicating ATExecSetTableSpaceNoStorage, > + * which should maybe be factored out to

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Alexey Kondratov
On 2021-01-21 04:41, Michael Paquier wrote: On Wed, Jan 20, 2021 at 03:34:39PM -0300, Alvaro Herrera wrote: On 2021-Jan-20, Alexey Kondratov wrote: Ugh, forgot to attach the patches. Here they are. Yeah, looks reasonable. + + if (changed) + /* Record dependency on

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-21 Thread Alexey Kondratov
On 2021-01-21 17:06, Alexey Kondratov wrote: On 2021-01-21 04:41, Michael Paquier wrote: There are no tests for partitioned tables, aka we'd want to make sure that the new partitioned index is on the correct tablespace, as well as all its leaves. It may be better to have at least two l

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-22 Thread Alexey Kondratov
On 2021-01-22 00:26, Justin Pryzby wrote: On Thu, Jan 21, 2021 at 11:48:08PM +0300, Alexey Kondratov wrote: Attached is a new patch set of first two patches, that should resolve all the issues raised before (ACL, docs, tests) excepting TOAST. Double thanks for suggestion to add more tests with

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-25 Thread Alexey Kondratov
On 2021-01-25 11:07, Michael Paquier wrote: On Fri, Jan 22, 2021 at 05:07:02PM +0300, Alexey Kondratov wrote: I have updated patches accordingly and also simplified tablespaceOid checks and assignment in the newly added SetRelTableSpace(). Result is attached as two separate patches for an ease

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-26 Thread Alexey Kondratov
On 2021-01-26 09:58, Michael Paquier wrote: On Mon, Jan 25, 2021 at 11:11:38PM +0300, Alexey Kondratov wrote: I updated comment with CCI info, did pgindent run and renamed new function to SetRelationTableSpace(). New patch is attached. [...] Yeah, all these checks we complicated from the

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-27 Thread Alexey Kondratov
On 2021-01-27 06:14, Michael Paquier wrote: On Wed, Jan 27, 2021 at 01:00:50AM +0300, Alexey Kondratov wrote: In the new 0002 I moved ACL check to the upper level, i.e. ExecReindex(), and removed expensive text generation in test. Not touched yet some of your previously raised concerns. Also

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-28 Thread Alexey Kondratov
On 2021-01-28 00:36, Alvaro Herrera wrote: On 2021-Jan-28, Alexey Kondratov wrote: I have read more about lock levels and ShareLock should prevent any kind of physical modification of indexes. We already hold ShareLock doing find_all_inheritors(), which is higher than ShareUpdateExclusiveLock

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-29 Thread Alexey Kondratov
On 2021-01-28 14:42, Alexey Kondratov wrote: On 2021-01-28 00:36, Alvaro Herrera wrote: I didn't look at the patch closely enough to understand why you're trying to do something like CLUSTER, VACUUM FULL or REINDEX without holding full AccessExclusiveLock on the relation. But do ke

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-01 Thread Alexey Kondratov
On 2021-01-30 05:23, Michael Paquier wrote: On Fri, Jan 29, 2021 at 08:56:47PM +0300, Alexey Kondratov wrote: On 2021-01-28 14:42, Alexey Kondratov wrote: No, you are right, we are doing REINDEX with AccessExclusiveLock as it was before. This part is a more specific one. It only applies to

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-02-03 Thread Alexey Kondratov
On 2021-02-03 09:37, Michael Paquier wrote: On Tue, Feb 02, 2021 at 10:32:19AM +0900, Michael Paquier wrote: On Mon, Feb 01, 2021 at 06:28:57PM +0300, Alexey Kondratov wrote: > Hm, IIUC, REINDEX CONCURRENTLY doesn't use either of them. It directly uses > index_create() wi

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-07-27 Thread Alexey Kondratov
patched Postgres binaries should be available in the PATH). It works well with master and fails with your patch applied. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company#!/usr/bin/env sh pg_ctl -D node1 stop > /dev/null pg_ctl -D node2

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-07-28 Thread Alexey Kondratov
On 2020-07-28 03:33, Andrey Lepikhov wrote: 27.07.2020 21:34, Alexey Kondratov пишет: Hi Andrey, On 2020-07-23 09:23, Andrey V. Lepikhov wrote: On 7/16/20 2:14 PM, Amit Langote wrote: Amit Langote EnterpriseDB: http://www.enterprisedb.com Version 5 of the patch. With changes caused by

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-09-01 Thread Alexey Kondratov
me how exactly. That way, I'd be glad if Michael could reword his explanation, so it'd more clear for me as well. BTW, I've started doing a review of the last patch set yesterday and will try to post some comments later. Regards -- Alexey Kondratov Postgres Profession

Re: Global snapshots

2020-09-04 Thread Alexey Kondratov
plements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it solves these errors handling issues above and tries to add proper comments everywhere. I think, that 0003 should be rebased on the top of it, or it could be a first patch in the set, since it may

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-08 Thread Alexey Kondratov
this check doesn't look costly. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Global snapshots

2020-09-08 Thread Alexey Kondratov
On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it solves these errors handling issues above and tries to add proper comments

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-08 Thread Alexey Kondratov
On 2020-09-08 17:00, Amit Langote wrote: Hi Alexey, On Tue, Sep 8, 2020 at 6:29 PM Alexey Kondratov wrote: On 2020-09-08 10:34, Amit Langote wrote: > Any thoughts on the taking out the refactoring changes out of the main > patch as I suggested? > +1 for splitting the patch. It w

Re: Global snapshots

2020-09-08 Thread Alexey Kondratov
On 2020-09-08 14:48, Fujii Masao wrote: On 2020/09/08 19:36, Alexey Kondratov wrote: On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-09 Thread Alexey Kondratov
On 2020-09-09 11:45, Andrey V. Lepikhov wrote: On 9/8/20 8:34 PM, Alexey Kondratov wrote: On 2020-09-08 17:00, Amit Langote wrote: wrote: On 2020-09-08 10:34, Amit Langote wrote: Another ambiguous part of the refactoring was in changing InitResultRelInfo() arguments: @@ -1278,6 +1280,7

Re: Global snapshots

2020-09-09 Thread Alexey Kondratov
On 2020-09-09 08:35, Masahiko Sawada wrote: On Wed, 9 Sep 2020 at 02:00, Alexey Kondratov wrote: On 2020-09-08 14:48, Fujii Masao wrote: > > IIUC, yes, the information required for automatic resolution is > WAL-logged and the standby tries to resolve those orphan transactions > fr

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-09-09 Thread Alexey Kondratov
es parenthesized option list. All the checks and validations are performed at the corresponding command code. This analyze_keyword is actually doing only an ANALYZE word normalization if it's used as an option. Why it could be harmful? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Global snapshots

2020-09-10 Thread Alexey Kondratov
On 2020-09-09 20:29, Fujii Masao wrote: On 2020/09/09 2:00, Alexey Kondratov wrote: According to the Sawada-san's v25 0002 the logic is pretty much the same there: +2. Pre-Commit phase (1st phase of two-phase commit) +3. Commit locally +Once we've prepared all of them,

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-09-18 Thread Alexey Kondratov
other places OK, changed, although I don't think it is important, since this is the only one tablespace variable there. *The patch did't applied cleanly http://cfbot.cputube.org/patch_24_2269.log Patch is rebased and attached with all the fixes described above. Regards -- A

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-09-19 Thread Alexey Kondratov
Hi Michael, Thank you for your comments. On 19.09.2019 7:43, Michael Paquier wrote: On Wed, Sep 18, 2019 at 03:46:20PM +0300, Alexey Kondratov wrote: Currently in Postgres SET TABLESPACE always comes with [ NOWAIT ] option, so I hope it worth adding this option here for convenience. Added in

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-09-19 Thread Alexey Kondratov
lespace of table, indexes or both? -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Conflict handling for COPY FROM

2019-09-20 Thread Alexey Kondratov
ignore_error = true; +            if (cstate->error_limit == -1) +                cstate->ignore_all_error = true; +        } If cstate->ignore_error is defined, then we have already processed options list, since this is the only one place, where it's set. So we should never get into this ereport, doesn't it? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company 1100.5 2420.1 300

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-09-24 Thread Alexey Kondratov
6b2a5c4de19f111ef24b63428033bb67%40postgrespro.ru Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company >From 4f06996f1e86dee389cb0f901cb83dba77c2abd8 Mon Sep 17 00:00:00 2001 From: Alexey Kondratov Date: Tue, 24 Sep 2019 12:29:57 +0300 Subject:

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-09-26 Thread Alexey Kondratov
On 01.08.2019 19:53, Alexey Kondratov wrote: On 26.07.2019 20:43, Liudmila Mantrova wrote: On a more general note, I wonder if everyone is happy with the --using-postgresql-conf option name, or we should continue searching for a narrower term. Unfortunately, I don't have any b

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-27 Thread Alexey Kondratov
standby is able to start with generated recovery configuration. [1] https://github.com/postgres/postgres/commit/d9f543e9e9be15f92abdeaf870e57ef289020191 Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company >From 8c607794f259cd4dec0fa6172b6

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-30 Thread Alexey Kondratov
de(); +   } Yeah, it makes sense. It is excessive for remote if we add '-R' there. I've updated and attached my test adding patch. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company >From b38bc7d71f7e7d68d66d3bf9af4e6371445aeab2

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-10-02 Thread Alexey Kondratov
g. Thus, it was definitely a good idea to add new tests. Two patches are attached: 1) First one fixes all the issues above; 2) Second one slightly increases pg_rewind overall code coverage from 74% to 78.6%. Should I put this fix on the next commitfest? Regards -- Alexey Kondratov Postgre

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-10-03 Thread Alexey Kondratov
On 03.10.2019 6:07, Michael Paquier wrote: On Wed, Oct 02, 2019 at 08:28:09PM +0300, Alexey Kondratov wrote: I've directly followed your guess and tried to elaborate pg_rewind test cases and... It seems I've caught a few bugs: 1) --dry-run actually wasn't completely 'dry&#x

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-10-04 Thread Alexey Kondratov
On 04.10.2019 11:37, Michael Paquier wrote: On Thu, Oct 03, 2019 at 12:43:37PM +0300, Alexey Kondratov wrote: On 03.10.2019 6:07, Michael Paquier wrote: I have reworked your first patch as per the attached. What do you think about it? The part with the control file needs to go down to v12

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-10-07 Thread Alexey Kondratov
On 07.10.2019 4:06, Michael Paquier wrote: On Fri, Oct 04, 2019 at 05:21:25PM +0300, Alexey Kondratov wrote: I've checked your patch, but it seems that it cannot be applied as is, since it e.g. adds a comment to 005_same_timeline.pl without actually changing the test. So I've slightl

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-10-22 Thread Alexey Kondratov
dff%40postgrespro.ru#98d06fefc88122385dacb2f03f7c30f7 Thanks for moving this patch forward! -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-07 Thread Alexey Kondratov
have marked it Waiting on Author. I hope I have addressed all issues in the new patch version which is attached. Also, I have added more detailed explanation of new functionality into the multi-line commit-message. Regards, -- Alexey Kondratov Postgres Professional https://www.postgres

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-27 Thread Alexey Kondratov
evels. However, if it was due to a signal, then some of next messages may be misleading, if e.g. user manually interrupted it for some reason. So that, I added a similar check here as well. Updated version of patch is attached. -- Alexey Kondratov Postgres Professional http

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-08-01 Thread Alexey Kondratov
wal instead of --using-postgresql-conf will be better? And --target-restore-command instead of --restore-command if we want to specify that this is restore_command for target server? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company >From 3

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-08-28 Thread Alexey Kondratov
y worker and each bgworker from the pool is a 'single producer --- single consumer' problem, then probably it is possible to wait and set/check flags without locks, but using just atomics. What do you think about this concept in general? Any concerns and criticism are welcome! Reg

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-08-29 Thread Alexey Kondratov
the WAL, it is safe to apply the sequentially. Am I wrong? Anyway, I'm going to double check the safety of this part later. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-11-12 Thread Alexey Kondratov
too small values of logical_decoding_work_mem. Probably it may be optimized, but I am not sure that it is worth doing right now. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Conflict handling for COPY FROM

2019-11-15 Thread Alexey Kondratov
re_all_error;    /* is error_limit -1 (ignore all error) +                                     * specified? */ Though it seems that we can just leave error_limit as a user-defined constant and track errors with something like errors_count. In that case you do not need auxiliary ignore_all_error flag. But probably i

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-11-20 Thread Alexey Kondratov
is specified. However, there are multiple code paths with table_open, so it becomes a bit ugly. I will leave the second patch aside for now and experiment with it. Actually, its main idea was to mimic ALTER INDEX ... SET TABLESPACE [NOWAIT] syntax, but probably it is better to stick wi

Re: Conflict handling for COPY FROM

2019-11-21 Thread Alexey Kondratov
On 18.11.2019 9:42, Surafel Temesgen wrote: On Fri, Nov 15, 2019 at 6:24 PM Alexey Kondratov mailto:a.kondra...@postgrespro.ru>> wrote: 1) Maybe it is fine, but now I do not like this part: +    portal = GetPortalByName(""); +    dest = CreateDestRec

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-11-27 Thread Alexey Kondratov
wo points (core part and error messages for pg_global), which are not clear for me right now. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company >From 6b76365b462c9332ced841bdfd0f80d862082156 Mon Sep 17 00:00:00 2001 From: Alexey Kondratov

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-12-02 Thread Alexey Kondratov
On 02.12.2019 11:21, Michael Paquier wrote: On Wed, Nov 27, 2019 at 08:47:06PM +0300, Alexey Kondratov wrote: The only difference is that point 3) and tablespace part of 5) were missing in RelationSetNewRelfilenode, so I added them, and I do 4) after 6) in REINDEX. Thus, it seems that in my

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-12-03 Thread Alexey Kondratov
On 01.12.2019 5:57, Michael Paquier wrote: On Thu, Sep 26, 2019 at 03:08:22PM +0300, Alexey Kondratov wrote: As Alvaro correctly pointed in the nearby thread [1], we've got an interference regarding -R command line argument. I agree that it's a good idea to reserve -R for recovery con

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-20 Thread Alexey Kondratov
, but still. We already have hundreds of GUCs and it is easy for a user to build a sub-optimal configuration, so does this overprotection make sense? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Physical replication slot advance is not persistent

2019-12-24 Thread Alexey Kondratov
ogRecPtrIsInvalid(endlsn))' and now pg_logical_replication_slot_advance and pg_physical_replication_slot_advance return InvalidXLogRecPtr if no-op. What do you think? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company P.S. CCed Simon and Michael as

Re: Physical replication slot advance is not persistent

2019-12-25 Thread Alexey Kondratov
On 25.12.2019 07:03, Kyotaro Horiguchi wrote: At Tue, 24 Dec 2019 20:12:32 +0300, Alexey Kondratov wrote in I dig into the code and it happens because of this if statement:     /* Update the on disk state when lsn was updated. */     if (XLogRecPtrIsInvalid(endlsn

Re: Physical replication slot advance is not persistent

2019-12-25 Thread Alexey Kondratov
On 25.12.2019 16:51, Alexey Kondratov wrote: On 25.12.2019 07:03, Kyotaro Horiguchi wrote: As the result I think what is needed there is just checking if the returned lsn is equal or larger than moveto. Doen't the following change work? -    if (XLogRecPtrIsInvalid(endlsn)) +    if (m

Re: Physical replication slot advance is not persistent

2019-12-26 Thread Alexey Kondratov
On 26.12.2019 11:33, Kyotaro Horiguchi wrote: At Wed, 25 Dec 2019 20:28:04 +0300, Alexey Kondratov wrote in Yep, it helps with physical replication slot persistence after advance, but the whole validation (moveto <= endlsn) does not make sense for me. The value of moveto should be >

Re: Physical replication slot advance is not persistent

2019-12-29 Thread Alexey Kondratov
On 2019-12-26 16:35, Alexey Kondratov wrote: Another concern is that ReplicationSlotIsDirty is added with the only one user. It also cannot be used by SaveSlotToPath due to the simultaneous usage of both flags dirty and just_dirtied there. In that way, I hope that we should call

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-01-04 Thread Alexey Kondratov
On 2019-12-02 11:21, Michael Paquier wrote: On Wed, Nov 27, 2019 at 08:47:06PM +0300, Alexey Kondratov wrote: Thus, I cannot get your point correctly here. Can you, please, elaborate a little bit more your concerns? The case of REINDEX CONCURRENTLY is pretty simple, because a new relation

Re: Physical replication slot advance is not persistent

2020-01-16 Thread Alexey Kondratov
On 09.01.2020 09:36, Kyotaro Horiguchi wrote: Hello. At Sun, 29 Dec 2019 15:12:16 +0300, Alexey Kondratov wrote in On 2019-12-26 16:35, Alexey Kondratov wrote: Another concern is that ReplicationSlotIsDirty is added with the only one user. It also cannot be used by SaveSlotToPath due to the

Re: Physical replication slot advance is not persistent

2020-01-28 Thread Alexey Kondratov
e, the term 'schedule' is used for VACUUM, constraint check or checkpoint itself. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

[Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-19 Thread Alexey Kondratov
ut nothing else came to mind. It is also possible to leave the only one option -- passing restore_command as command line argument. What do you think? -- Alexey Kondratov Postgres Professional: https://www.postgrespro.com Russian Postgres Company diff --combined src/bin/pg_rewind

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-22 Thread Alexey Kondratov
e of IFDEF and mocking up ereport for frontend, which is a bit ugly. -- Alexey Kondratov Postgres Professional: https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-25 Thread Alexey Kondratov
/flat/CA%2BU5nMKyuDxr0%3D5PSen1DZJndauNdz8BuSREau%3DScN-7DZ9acA%40mail.gmail.com -- Alexey Kondratov Postgres Professional:https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-29 Thread Alexey Kondratov
Hi Andrey, Will you add this patch to CF? I'm going to review it. Best regards, Andrey Borodin Here it is https://commitfest.postgresql.org/20/1849/ -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-29 Thread Alexey Kondratov
sn't touch any data files. So I guess it behaves exactly as you described and we do not need a separate tool. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-11-07 Thread Alexey Kondratov
On 30.10.2018 06:01, Michael Paquier wrote: On Mon, Oct 29, 2018 at 12:09:21PM +0300, Alexey Kondratov wrote: Currently in the patch, with dry-run option (-n) pg_rewind only fetches missing WALs to be able to build file map, while doesn't touch any data files. So I guess it behaves exact

Re: Conflict handling for COPY FROM

2020-03-10 Thread Alexey Kondratov
works just fine. [1] https://github.com/postgres/postgres/blob/0a42a2e9/src/backend/tcop/postgres.c#L1178 [2] https://github.com/postgres/postgres/blob/0a42a2e9/src/backend/tcop/postgres.c#L1226 Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-12 Thread Alexey Kondratov
which is frontend-only. Is it there for historical reasons only or something else? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-12 Thread Alexey Kondratov
Hi Justin, On 09.03.2020 23:04, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 08:53:04AM -0600, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 03:35:27PM +0300, Alexey Kondratov wrote: Anyway, new version is attached. It is rebased in order to resolve conflicts with a recent fix of REINDEX

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Alexey Kondratov
On 2020-03-26 02:40, Justin Pryzby wrote: On Thu, Mar 12, 2020 at 08:08:46PM +0300, Alexey Kondratov wrote: On 09.03.2020 23:04, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 08:53:04AM -0600, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 03:35:27PM +0300, Alexey Kondratov wrote: tests for that

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Alexey Kondratov
to rewrite VACUUM's part again with OptTableSpace. Maybe I will manage it this time. I will take into account all your text edits as well. Thanks -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-26 Thread Alexey Kondratov
have tried to do what you proposed and attached is a small patch, that introduces xlogarchive.h. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/backend/access/transam/timeline.c b/src/backend/access/transam/timeline.c ind

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-30 Thread Alexey Kondratov
Let us use camel case 'restoreCommand' here as in the header for tidiness. I have left 0001 intact, but fixed all these small remarks in the 0002. Please, find it attached. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company From f

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-30 Thread Alexey Kondratov
ng with syntax as it is in 0001-0002 2) Implement tail syntax for VACUUM, but with limitation for VACUUM FULL of the entire database + TABLESPACE change 3) Change TABLESPACE to a fully reserved word Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-31 Thread Alexey Kondratov
On 2020-03-30 21:34, Justin Pryzby wrote: On Mon, Mar 30, 2020 at 09:02:22PM +0300, Alexey Kondratov wrote: Hmm, I went through the well known to me SQL commands in Postgres and a bit more. Parenthesized options list is mostly used in two common cases: There's also ANALYZE(VERBOSE), RE

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-04-01 Thread Alexey Kondratov
for doing that! And to all others who participated. -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-03 Thread Alexey Kondratov
It seems to be equivalent to pg_sleep, doesn't it? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Company

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-06 Thread Alexey Kondratov
d idea. Yes, it saves us a few checks here and there now, but it may reduce a further reusability of these internal routines in the future. XXX: for cluster/vacuum, it might be more friendly to check before clustering the table, rather than after clustering and re-indexing.

Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-19 Thread Alexey Kondratov
t required TCP for stable running. That way, if it really could happen why not to just skip binding to 127.0.0/24 addresses other than 127.0.0.1 outside of Linux/Windows as per attached patch_PostgresNode.diff? Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com R

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-20 Thread Alexey Kondratov
ss. Regards -- Alexey Kondratov Postgres Professional https://www.postgrespro.com Russian Postgres Companydiff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm index db47a97d196..f7b488ed464 100644 --- a/src/test/perl/PostgresNode.pm +++ b/src/test/perl/PostgresNode.pm @@ -119

  1   2   >