Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-26 Thread Fujii Masao
On 2020/11/27 0:15, Bharath Rupireddy wrote: On Thu, Nov 26, 2020 at 7:37 PM Fujii Masao wrote: What do you mean by normal shutdown of bgworker? Is it that bgworker has exited successfully with exit code 0 or for some reason with exit code other than 0? Or is it when the postmaster is

Re: autovac issue with large number of tables

2020-11-26 Thread Fujii Masao
nk you for updating the patch! I'll also run the performance test you shared with the latest version patch. +1 Thank you! It's very helpful. Agreed. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-26 Thread Fujii Masao
On 2020/11/27 1:45, Matthias van de Meent wrote: On Thu, 26 Nov 2020 at 00:55, Fujii Masao wrote: +* A heap scan need not return tuples for the last page it has +* scanned. To ensure that heap_blks_scanned is equivalent to

Re: Allow some recovery parameters to be changed with reload

2020-11-26 Thread Fujii Masao
On 2020/11/27 9:30, Kyotaro Horiguchi wrote: At Thu, 26 Nov 2020 22:43:48 +0900, Fujii Masao wrote in On 2020/11/12 4:38, Sergei Kornilov wrote: Hello Anyway, for now I think that your first patch would be enough, i.e., just change the context of restore_command to PGC_SIGHUP. Glad

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-26 Thread Fujii Masao
custom SIGTERM handler worker_spi_sigterm() with die() and custom SIGHUP handler worker_spi_sighup() with standard SignalHandlerForConfigReload() Yeah, I pushed them. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-27 Thread Fujii Masao
On 2020/11/27 1:51, Fujii Masao wrote: On 2020/11/27 1:45, Matthias van de Meent wrote: On Thu, 26 Nov 2020 at 00:55, Fujii Masao wrote: +    * A heap scan need not return tuples for the last page it has +    * scanned. To ensure

Re: autovac issue with large number of tables

2020-11-27 Thread Fujii Masao
On 2020/11/27 18:38, Kasahara Tatsuhito wrote: Hi, On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao wrote: On 2020/11/26 10:41, Kasahara Tatsuhito wrote: On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada wrote: On Wed, Nov 25, 2020 at 4:18 PM Kasahara Tatsuhito wrote: Hi, On Wed, Nov

Re: Feature improvement for pg_stat_statements

2020-11-27 Thread Fujii Masao
s are removed. But it's called even in other cases. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-29 Thread Fujii Masao
On 2020/11/27 16:47, Bharath Rupireddy wrote: On Fri, Nov 27, 2020 at 12:26 PM Fujii Masao wrote: > > When I read the patch again, I found that, with the patch, the shutdown > > of worker_spi causes to report the following FATAL message. > > > > FATAL:

Re: Fix typo in cost.h

2020-11-29 Thread Fujii Masao
On 2020/11/30 12:15, Tang, Haiying wrote: Hi Found a possible typo in cost.h -/* If you change these, update backend/utils/misc/postgresql.sample.conf */ +/* If you change these, update backend/utils/misc/postgresql.conf.sample */ Pushed. Thanks! Regards, -- Fujii Masao Advanced

Re: Feature improvement for pg_stat_statements

2020-11-29 Thread Fujii Masao
On 2020/11/30 12:08, Seino Yuki wrote: 2020-11-27 22:39 に Fujii Masao さんは書きました: On 2020/11/27 21:39, Seino Yuki wrote: 2020-11-27 21:37 に Seino Yuki さんは書きました: 2020-11-16 12:28 に Seino Yuki さんは書きました: Due to similar fixed, we have also merged the patches discussed in the following thread

Re: autovac issue with large number of tables

2020-11-30 Thread Fujii Masao
On 2020/11/30 10:43, Masahiko Sawada wrote: On Sun, Nov 29, 2020 at 10:34 PM Kasahara Tatsuhito wrote: Hi, Thanks for you comments. On Fri, Nov 27, 2020 at 9:51 PM Fujii Masao wrote: On 2020/11/27 18:38, Kasahara Tatsuhito wrote: Hi, On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao

Re: Feature improvement for pg_stat_statements

2020-11-30 Thread Fujii Masao
On 2020/11/30 23:05, Seino Yuki wrote: 2020-11-30 15:02 に Fujii Masao さんは書きました: On 2020/11/30 12:08, Seino Yuki wrote: 2020-11-27 22:39 に Fujii Masao さんは書きました: On 2020/11/27 21:39, Seino Yuki wrote: 2020-11-27 21:37 に Seino Yuki さんは書きました: 2020-11-16 12:28 に Seino Yuki さんは書きました: Due to

Re: Add Information during standby recovery conflicts

2020-11-30 Thread Fujii Masao
On 2020/11/20 18:17, Drouvot, Bertrand wrote: Hi, On 11/17/20 4:44 PM, Fujii Masao wrote: Thanks for updating the patch! Here are review comments. +    Controls whether a log message is produced when the startup process +    is waiting longer than deadlock_timeout +    for

Re: Add Information during standby recovery conflicts

2020-11-30 Thread Fujii Masao
other words, how should we handle the case where nprocs == 0 (i.e., nprocs has not been incrmented at all)? This would very rarely happen, but can happen. In this case, since buf.data is empty, at least there seems no need to log the list of conflicting processes in detail message. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: A failure of standby to follow timeline switch

2021-01-11 Thread Fujii Masao
catead like the existing regression test for timeline switch in 004_timeline_switch.pl does, instead of finding the specific message in the log file. I attached the POC patch. Thought? Regards, -- Fujii Masao v5-0001-Move-TAP-log-searching-feature-to-common-modules.patch Description: Binary data

Re: [PATCH] Feature improvement for CLOSE, FETCH, MOVE tab completion

2021-01-11 Thread Fujii Masao
> Thanks for your comment. Agreed. > > So regarding the tab completion for DECLARE statement, perhaps it > would be better to follow the documentation? IMO yes because it's less confusing to make the document and tab-completion consistent. Regards, -- Fujii Masao

Re: outdated references to replication timeout

2021-01-12 Thread Fujii Masao
e about walsender timeout, I also want to change the message about walreceiver timeout, so as to make them more consistent. For example, - (errmsg("terminating walreceiver due to timeout"))); + (errmsg("terminating WAL receiver process due to WAL receiver timeout"))); Regards, -- Fujii Masao

Re: A failure of standby to follow timeline switch

2021-01-12 Thread Fujii Masao
On Wed, Jan 13, 2021 at 10:48 AM Kyotaro Horiguchi wrote: > > At Tue, 12 Jan 2021 10:47:21 +0900, Fujii Masao wrote > in > > On Sat, Jan 9, 2021 at 5:08 AM Alvaro Herrera > > wrote: > > > > > > Masao-san: Are you intending to act as committer for these?

Re: [PATCH] Feature improvement for CLOSE, FETCH, MOVE tab completion

2021-01-12 Thread Fujii Masao
On Tue, Jan 12, 2021 at 11:09 AM Fujii Masao wrote: > > On Tue, Jan 12, 2021 at 10:00 AM Masahiko Sawada > wrote: > > > > On Mon, Jan 11, 2021 at 11:00 PM Peter Eisentraut > > wrote: > > > > > > On 2021-01-05 10:56, Masahiko Sawada wrote: > >

Re: Add Information during standby recovery conflicts

2021-01-13 Thread Fujii Masao
On Fri, Jan 8, 2021 at 2:15 PM Drouvot, Bertrand wrote: > > Hi, > > On 1/8/21 5:19 AM, Fujii Masao wrote: > > > > On 2021/01/08 11:17, Kyotaro Horiguchi wrote: > >> At Fri, 8 Jan 2021 01:32:11 +0900, Fujii Masao > >> wrote in > >>> > >

Re: Add Information during standby recovery conflicts

2021-01-13 Thread Fujii Masao
On Fri, Jan 8, 2021 at 3:43 PM Drouvot, Bertrand wrote: > > > On 1/8/21 7:24 AM, Fujii Masao wrote: > > CAUTION: This email originated from outside of the organization. Do > > not click links or open attachments unless you can confirm the sender > > a

Re: outdated references to replication timeout

2021-01-13 Thread Fujii Masao
On Wed, Jan 13, 2021 at 10:51 PM John Naylor wrote: > > > On Tue, Jan 12, 2021 at 9:37 PM Fujii Masao wrote: > > > > Thanks for the patch! I think this change makes sense. > > > > - (errmsg("terminating walsender

Re: A failure of standby to follow timeline switch

2021-01-13 Thread Fujii Masao
On Thu, Jan 14, 2021 at 10:10 AM Kyotaro Horiguchi wrote: > > At Wed, 13 Jan 2021 16:51:32 -0300, Alvaro Herrera > wrote in > > On 2021-Jan-13, Fujii Masao wrote: > > > > > Thanks for the review! > > > I'm ok with this change (i.e., insert only single

Re: [PATCH] Feature improvement for CLOSE, FETCH, MOVE tab completion

2021-01-13 Thread Fujii Masao
On 2021/01/14 14:38, Masahiko Sawada wrote: On Wed, Jan 13, 2021 at 1:55 PM Fujii Masao wrote: On Tue, Jan 12, 2021 at 11:09 AM Fujii Masao wrote: On Tue, Jan 12, 2021 at 10:00 AM Masahiko Sawada wrote: On Mon, Jan 11, 2021 at 11:00 PM Peter Eisentraut wrote: On 2021-01-05 10:56

Re: Outdated replication protocol error?

2021-01-13 Thread Fujii Masao
en in that case. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [bug fix] Fix the size calculation for shmem TOC

2021-01-14 Thread Fujii Masao
patch looks good to me. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-14 Thread Fujii Masao
unction to be registered as a callback more than once. This is a bug. +CREATE FUNCTION postgres_fdw_disconnect () Do we really want postgres_fdw_disconnect() with no argument? IMO postgres_fdw_disconnect() with the server name specified is enough. But I'd like to hear the opinion about tha

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-14 Thread Fujii Masao
On 2021/01/14 20:36, Bharath Rupireddy wrote: On Thu, Jan 14, 2021 at 3:52 PM Fujii Masao wrote: - if (!HeapTupleIsValid(tup)) - elog(ERROR, "cache lookup failed for user mapping %u", entry->key); - umform = (Form_pg_user_mapping) GETSTRUCT(tup); -

Re: [bug fix] Fix the size calculation for shmem TOC

2021-01-14 Thread Fujii Masao
On 2021/01/14 18:15, Fujii Masao wrote: On 2021/01/14 17:47, tsunakawa.ta...@fujitsu.com wrote: Hello, The attached patch fixes a trivial mistake in the calculation of shmem TOC.  The original code allocates unduly large memory because it adds the result of add_size() to its left

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Fujii Masao
g related code to disconnect_cached_connections from postgres_fdw_disconnect. Please review v11 further. Thanks for updating the patch! The patch for postgres_fdw_get_connections() basically looks good to me. So at first I'd like to push it. Attached is the patch that I extracted postgres_fdw

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Fujii Masao
On 2021/01/18 13:46, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 9:38 AM Fujii Masao wrote: Please review v11 further. Thanks for updating the patch! The patch for postgres_fdw_get_connections() basically looks good to me. So at first I'd like to push it. Attached is the patch

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Fujii Masao
ction is used in the current local + transaction but its foreign server or user mapping is changed or + dropped Or better description? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/18 15:37, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 11:58 AM Fujii Masao wrote: On 2021/01/18 15:02, Hou, Zhijie wrote: We need to create the loopback3 with user mapping public, otherwise the test might become unstable as shown below. Note that loopback and loopback2

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/18 23:14, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 11:44 AM Fujii Masao wrote: I will post patches for the other function postgres_fdw_disconnect, GUC and server level option later. Thanks! Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() function, 0002

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/18 22:03, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 6:17 PM Fujii Masao wrote: +1 to add it after "dropped (Note )", how about as follows with slight changes? dropped (Note that server name of an invalid connection can be NULL if the server is dropped)

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
." Sounds good to me. So patch attached. +1 Thanks! I pushed the patch. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Stronger safeguard for archive recovery not to miss data

2021-01-19 Thread Fujii Masao
e primary, or turn off hot_standby here."))); With the patch, we never reach the above code? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-19 Thread Fujii Masao
On 2021/01/19 12:09, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 9:11 PM Fujii Masao wrote: Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() function, 0002 is for keep_connections GUC and 0003 is for keep_connection server level option. Thanks! Please review it

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/20 17:41, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 11:53 AM Fujii Masao wrote: So, furthermore, we can use hash_search() to find the target cached connection, instead of using hash_seq_search(), when the server name is given. This would simplify the code a bit more? Of

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/20 19:17, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 3:24 PM Fujii Masao wrote: Keeping above in mind, I feel we can do hash_seq_search(), as we do currently, even when the server name is given as input. This way, we don't need to bother much on the above points. Tho

Re: adding wait_start column to pg_locks

2021-01-20 Thread Fujii Masao
ck acquired. There seems the case where the wait start time is NULL even when "grant" is false. It's better to add note about that case into the docs? For example, I found that the wait start time is NULL while the startup process is waiting for the lock. Is this only that case?

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/21 12:00, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 6:58 PM Fujii Masao wrote: + * It checks if the cache has a connection for the given foreign server that's + * not being used within current transaction, then returns true. If the + * connection is in use, then it em

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/21 14:46, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao wrote: > >> + if (entry->server_hashvalue == hashvalue && + (entry->x

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Fujii Masao
On 2021/01/21 16:16, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 12:17 PM Fujii Masao wrote: On 2021/01/21 14:46, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao wrote: > >> + if (entry->server_hashvalue

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Fujii Masao
On 2021/01/22 1:17, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: My opinion is to check "!all", but if others prefer using such boolean flag, I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough there, my earlier und

Re: adding wait_start column to pg_locks

2021-01-22 Thread Fujii Masao
On 2021/01/22 14:37, torikoshia wrote: On 2021-01-21 12:48, Fujii Masao wrote: Thanks for updating the patch! I think that this is really useful feature!! Thanks for reviewing! I have two minor comments. +  +   wait_start timestamptz The column name "wait_start"

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-22 Thread Fujii Masao
On 2021/01/22 3:29, Fujii Masao wrote: On 2021/01/22 1:17, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: My opinion is to check "!all", but if others prefer using such boolean flag, I'd withdraw my opinion. I'm really sorry, actually

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-24 Thread Fujii Masao
On 2021/01/23 13:40, Bharath Rupireddy wrote: On Fri, Jan 22, 2021 at 6:43 PM Fujii Masao wrote: Please review the v16 patch set further. Thanks! Will review that later. + /* +* For the given server, if we closed connection or it is still

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/25 18:13, Bharath Rupireddy wrote: On Mon, Jan 25, 2021 at 1:20 PM Fujii Masao wrote: Yeah, connections can be discarded by non-super users using postgres_fdw_disconnect_all and postgres_fdw_disconnect. Given the fact that a non-super user requires a password to access foreign

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/25 19:28, Bharath Rupireddy wrote: On Mon, Jan 25, 2021 at 3:17 PM Fujii Masao wrote: Yes, if required backends can establish the connection again. But my worry is this - a non-super user disconnecting all or a given connection created by a super user? Yes, I was also worried

Re: adding wait_start column to pg_locks

2021-01-25 Thread Fujii Masao
On 2021/01/22 18:11, Fujii Masao wrote: On 2021/01/22 14:37, torikoshia wrote: On 2021-01-21 12:48, Fujii Masao wrote: Thanks for updating the patch! I think that this is really useful feature!! Thanks for reviewing! I have two minor comments. +  +   wait_start timestamptz

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
x27;t change any logic, but I updated some comments and docs. Also I added the regresssion test to check that postgres_fdw_disconnect() closes multiple connections. Barring any objection, I will commit this version. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Develo

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 12:08, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 12:38 AM Fujii Masao wrote: Attaching v17 patch set, please review it further. Thanks for updating the patch! Attached is the tweaked version of the patch. I didn't change any logic, but I updated some comment

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 16:05, Tom Lane wrote: Fujii Masao writes: Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy ... Yes I'm investigating that. -- Return false as connections are still in use, warnings are issued. SELECT postgres_fdw_disconnec

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 16:33, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 12:54 PM Fujii Masao wrote: On 2021/01/26 16:05, Tom Lane wrote: Fujii Masao writes: Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy ... Yes I'm investigating

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 16:39, Fujii Masao wrote: On 2021/01/26 16:33, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 12:54 PM Fujii Masao wrote: On 2021/01/26 16:05, Tom Lane wrote: Fujii Masao writes: Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-26 Thread Fujii Masao
On 2021/01/26 17:07, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 1:27 PM Fujii Masao wrote: Yes, so I pushed that change to stabilize the regression test. Let's keep checking how the results of buildfarm members are changed. Sorry, I'm unfamiliar with checking the system

Re: Transactions involving multiple postgres foreign servers, take 2

2021-01-26 Thread Fujii Masao
ut seems not right theoretically. This makes me wonder if new FDW API is not good at handling the case where some operations need to be performed once per transaction end. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] remove pg_standby

2021-01-27 Thread Fujii Masao
everal releases, and then drop pg_standby. This seems safe because there might be some users. While it's been marked as obsolete, maybe WAL prefetch feature doesn't work with pg_standby, but we can live with that because it's obsolete. Regards, -- Fujii Masao Advanced Computi

Re: protect pg_stat_statements_info() for being used without the library loaded

2021-01-27 Thread Fujii Masao
I will apply it on HEAD. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
) so that it doesn't close the connection at all, ISTM that the results of postgres_fdw_get_connections() would not be stable because entry->invalidated would vary based on whether CLOBBER_CACHE_ALWAYS is used or not. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 14:12, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao wrote: On 2021/01/29 11:09, Tom Lane wrote: Bharath Rupireddy writes: On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=trilobite&dt=202

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 14:53, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao wrote: BTW, even if we change pgfdw_inval_callback() so that it doesn't close the connection at all, ISTM that the results of postgres_fdw_get_connections() would not be stable because

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 14:46, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 11:08 AM Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao wrote: On 2021/01/29 14:12, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao wrote: On 2021/01/29 11:09, Tom Lane

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 15:44, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 11:54 AM Fujii Masao wrote: IIRC, when we were finding a way to close the invalidated connections so that they don't leaked, we had two options: 1) let those connections (whether currently being used in the xact o

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 16:12, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 12:36 PM Fujii Masao wrote: On 2021/01/29 15:44, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 11:54 AM Fujii Masao wrote: IIRC, when we were finding a way to close the invalidated connections so that they don&#

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Fujii Masao
On 2021/01/29 19:45, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 1:24 PM Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 1:17 PM Fujii Masao wrote: But if the issue is only the inconsistency of test results, we can go with the option (2)? Even with (2), we can make the test stable

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-31 Thread Fujii Masao
On 2021/01/30 9:28, Bharath Rupireddy wrote: On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: + /* +* It doesn't make sense to show this entry in the output with a +* NULL server_name as it will be closed at the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-31 Thread Fujii Masao
tion into the foreign server. But I'm not sure if it's good idea to expose the option as GUC. Also if there is the consensus about this, please correct me. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-01 Thread Fujii Masao
On 2021/02/01 16:13, Bharath Rupireddy wrote: On Mon, Feb 1, 2021 at 12:29 PM Fujii Masao wrote: On 2021/01/30 9:28, Bharath Rupireddy wrote: On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: + /* +* It doesn't make sense to show this

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-01 Thread Fujii Masao
On 2021/02/01 16:39, Bharath Rupireddy wrote: On Mon, Feb 1, 2021 at 12:43 PM Fujii Masao wrote: On 2021/01/27 10:06, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 8:38 AM Bharath Rupireddy wrote: I will post "keep_connections" GUC and "keep_connection" server

Re: Transactions involving multiple postgres foreign servers, take 2

2021-02-02 Thread Fujii Masao
On 2021/01/27 14:08, Masahiko Sawada wrote: On Wed, Jan 27, 2021 at 10:29 AM Fujii Masao wrote: You fixed some issues. But maybe you forgot to attach the latest patches? Yes, I've attached the updated patches. Thanks for updating the patch! I tried to review 0001 and 0002 as the

Re: adding wait_start column to pg_locks

2021-02-02 Thread Fujii Masao
On 2021/02/02 22:00, torikoshia wrote: On 2021-01-25 23:44, Fujii Masao wrote: Another comment is; Doesn't the change of MyProc->waitStart need the lock table's partition lock? If yes, we can do that by moving LWLockRelease(partitionLock) just after the change of MyProc->wait

Re: adding wait_start column to pg_locks

2021-02-02 Thread Fujii Masao
On 2021/02/03 1:49, Fujii Masao wrote: On 2021/02/02 22:00, torikoshia wrote: On 2021-01-25 23:44, Fujii Masao wrote: Another comment is; Doesn't the change of MyProc->waitStart need the lock table's partition lock? If yes, we can do that by moving LWLockRelease(partitionLoc

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-03 Thread Fujii Masao
On 2021/02/03 13:56, Bharath Rupireddy wrote: On Tue, Feb 2, 2021 at 9:45 AM Fujii Masao wrote: One merit of keep_connections that I found is that we can use it even when connecting to the older PostgreSQL that doesn't support idle_session_timeout. Also it seems simpler t

Re: adding wait_start column to pg_locks

2021-02-05 Thread Fujii Masao
On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms. So spinlock is necessary when updating "waitStart" without holding the partition lock? Also GetLockStatusData() needs spinlock when reading "waitSt

Re: There doesn't seem to be any case where PQputCopyEnd() returns 0

2021-02-05 Thread Fujii Masao
on [1] about this issue. [1] https://www.postgresql.org/message-id/CA+Tgmobjj+0modbnmjy7ezeBFOBo9d2mAVcSPkzLx4LtZmc==g...@mail.gmail.com Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-02-07 Thread Fujii Masao
uot; should be used instead of "=" when updating m_wal_write_time? + INSTR_TIME_SET_CURRENT(duration); + INSTR_TIME_SUBTRACT(duration, start); + WalStats.m_wal_sync_time = INSTR_TIME_GET_MICROSEC(duration); Also "=" should be "

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-02-07 Thread Fujii Masao
On 2021/02/08 13:01, Fujii Masao wrote: On 2021/02/05 8:45, Masahiro Ikeda wrote: I pgindented the patches. Thanks for updating the patches! +   XLogWrite, which nomally called by an +   issue_xlog_fsync, which nomally called by an Typo: "nomally" should be

Re: adding wait_start column to pg_locks

2021-02-09 Thread Fujii Masao
On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms. So spinlock is necessary when updating "waitStart" without holding the

Re: adding wait_start column to pg_locks

2021-02-09 Thread Fujii Masao
On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms. So spinlock is necessary when updating

Re: adding wait_start column to pg_locks

2021-02-09 Thread Fujii Masao
On 2021/02/09 19:11, Fujii Masao wrote: On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms

ERROR: invalid spinlock number: 0

2021-02-09 Thread Fujii Masao
ore the startup of walreceiver can cause the error. Also pg_stat_get_wal_receiver() calls pg_atomic_read_u64() while a spinlock is being held. Maybe this may cause the process to get stuck in --disable-atomics case, I guess. Thought? Regards, -- Fujii Masao Advanced Computing Technology Center R

Re: adding wait_start column to pg_locks

2021-02-09 Thread Fujii Masao
On 2021/02/09 23:31, torikoshia wrote: On 2021-02-09 22:54, Fujii Masao wrote: On 2021/02/09 19:11, Fujii Masao wrote: On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021

Re: ERROR: invalid spinlock number: 0

2021-02-11 Thread Fujii Masao
On 2021/02/11 21:55, Michael Paquier wrote: Hi Fujii-san, On Tue, Feb 09, 2021 at 11:17:04PM +0900, Fujii Masao wrote: ISTM that the commit 2c8dd05d6c caused this issue. The commit changed pg_stat_get_wal_receiver() so that it reads "writtenUpto" by using pg_atomic_read_u64().

Re: issue in pgfdw_report_error()?

2021-12-03 Thread Fujii Masao
On 2021/11/22 13:59, Bharath Rupireddy wrote: On Mon, Nov 22, 2021 at 8:17 AM Fujii Masao wrote: Well, in that case, why can't we get rid of "(message_primary != NULL" and just have "message_primary[0] != '\0' ? errmsg_internal("%s", message_pri

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-12-03 Thread Fujii Masao
sult() can determine which log message to report. Probably we can add new boolean argument to pgfdw_get_cleanup_result() so that it should be set to true for cancel request case, but false for query case. Then pgfdw_get_cleanup_result() can decide wihch message to log based on that argument. But i

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-12-03 Thread Fujii Masao
included in the patch. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Allow escape in application_name

2021-12-03 Thread Fujii Masao
+ dbname = "[unknown]"; I'm still not sure why these are necessary. Could you clarify that? + Same as , this is a + printf-style string. Accepted escapes are + bit different from , + but padding can be used like a

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-12-06 Thread Fujii Masao
what I proposed before. Patch attached. This patch changes pgfdw_exec_cleanup_query() so that it tells its callers the information about whether the timeout expired or not. Then the callers (pgfdw_exec_cleanup_query and pgfdw_cancel_query) report the warning messages based on the results from pgfdw_exec_cleanup_query

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-12-08 Thread Fujii Masao
On 2021/12/06 20:50, Bharath Rupireddy wrote: On Mon, Dec 6, 2021 at 1:47 PM Fujii Masao wrote: Yeah, I agree that's not elegant.. So I'd like to propose new patch with different design from what I proposed before. Patch attached. This patch changes pgfdw_exec_cleanup_query()

Re: Allow escape in application_name

2021-12-08 Thread Fujii Masao
on_name is set to application_name of a pgfdw + connection after placeholder conversion, thus the resulting string is + subject to the same restrictions alreadby mentioned above. This description doesn't need to be added if 0001 patch is applied, is it? Because 0001 patch adds very simi

Re: Allow escape in application_name

2021-12-09 Thread Fujii Masao
ng value of application_name in local server" etc? I agree that it's good idea to add the example about how escape sequences are replaced. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Allow escape in application_name

2021-12-15 Thread Fujii Masao
ed version of 0002 patch. I applied some cosmetic changes, improved comments and docs. Could you review this version? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATIONdiff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src

Re: Allow escape in application_name

2021-12-15 Thread Fujii Masao
es/m/848ff477-effd-42b9-8b25-3f7cfe289...@oss.nttdata.com Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-16 Thread Fujii Masao
mpilation failure. guc.c:5453:4: error: implicit declaration of function 'EmitErrorsOnPlaceholders' is invalid in C99 [-Werror,-Wimplicit-function-declaration] EmitErrorsOnPlaceholders(placeholder); ^ - ereport(WARNING, + erepo

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-16 Thread Fujii Masao
also reported the failure of regression test for your patch. [1] http://commitfest.cputube.org/ Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Allow escape in application_name

2021-12-16 Thread Fujii Masao
than "[unknown]" when appname or username, etc was NULL or '\0'. To implement this behavior, I argued to remove the check itself. But there are several options: #1. use "[unknown]" #2. add the check but not use "[unknown]" #3. don't add the check (i.e

Re: sequences vs. synchronous replication

2021-12-21 Thread Fujii Masao
e similar check is performed at the beginning of SyncRepWaitForLSN(), so probably we can reuse that code. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

<    1   2   3   4   5   6   7   8   9   10   >