Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-04 Thread Fujii Masao
On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao wrote: On 2021/03/03 14:33, Masahiro Ikeda wrote: On 2021-02-24 16:14, Fujii Masao wrote

Re: [PATCH] pgbench: Bug fix for the -d option

2021-03-04 Thread Fujii Masao
dbName = env; else - dbName = ""; + dbName = get_user_name_or_exit(progname); If dbName is set by libpq, the above also is not necessary? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Get memory contexts of an arbitrary backend process

2021-03-04 Thread Fujii Masao
seems to me that the architecture of the requestor waiting for the dumper leads to this problem and complicates things. Considering the discussion printing backtrace discussion[1], it seems reasonable that the requestor just sends a signal and dumper dumps to the log file. +1 Regards, -- Fujii

Re: n_mod_since_analyze isn't reset at table truncation

2021-03-04 Thread Fujii Masao
On 2021/03/04 12:40, Julien Rouhaud wrote: On Thu, Mar 04, 2021 at 12:21:14PM +0900, Fujii Masao wrote: On 2021/03/04 11:24, Julien Rouhaud wrote: On Thu, Mar 04, 2021 at 10:35:19AM +0900, Masahiko Sawada wrote: While reviewing "autoanalyze on partitioned table" patch, I rea

Re: Is it useful to record whether plans are generic or custom?

2021-03-05 Thread Fujii Masao
plan or custom plan is used? If so, it's better to expose seq_scan (num of sequential scans processed by the query) and idx_scan (num of index scans processed by the query) like pg_stat_all_tables, per query in pg_stat_statements? Regards, -- Fujii Masao Advanced Computing Technology Center Re

Re: Is it useful to record whether plans are generic or custom?

2021-03-05 Thread Fujii Masao
e AS SELECT * FROM pgbench_accounts WHERE aid = $1 3 | 0 | EXPLAIN ANALYZE EXECUTE hoge(1) Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: n_mod_since_analyze isn't reset at table truncation

2021-03-05 Thread Fujii Masao
On 2021/03/05 15:59, Julien Rouhaud wrote: On Fri, Mar 05, 2021 at 03:31:33PM +0900, Fujii Masao wrote: On 2021/03/04 12:40, Julien Rouhaud wrote: In that case, conversely, we want to trigger autoanalyze ASAP because the contents in the table was changed very much? We might want, but

Re: [PATCH] pgbench: Bug fix for the -d option

2021-03-05 Thread Fujii Masao
On 2021/03/05 16:33, Michael Paquier wrote: On Fri, Mar 05, 2021 at 01:30:11PM +0900, Fujii Masao wrote: if ((env = getenv("PGDATABASE")) != NULL && *env != '\0') dbName = env; - else if (lo

Re: shared-memory based stats collector

2021-03-05 Thread Fujii Masao
-- */ The definition of PgArchiverMain() should be placed just after the above comment. exit(0) in PgArchiverMain() should be proc_exit(0)? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-05 Thread Fujii Masao
On 2021/03/05 22:45, David Steele wrote: Hi Fujii, On 9/8/20 1:17 PM, James Coleman wrote: On Tue, Aug 18, 2020 at 12:25 PM Fujii Masao wrote: Thanks for updating the patch! But it failed to be applied to the master branch cleanly because of the recent commit 0038f94387. So could you

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-07 Thread Fujii Masao
know that state. - When receiving that signal, postmaster needs to move its state to new state (e.g., PM_CONSISTENT). Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-07 Thread Fujii Masao
On 2021/03/05 19:54, Masahiro Ikeda wrote: On 2021-03-05 12:47, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao

Re: [PATCH] pgbench: improve \sleep meta command

2021-03-07 Thread Fujii Masao
ine, my_command->argv[0], +"\\sleep command argument must be an integer", I like the error message like "invalid sleep time, must be an integer". Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-08 Thread Fujii Masao
On 2021/03/08 19:42, Masahiro Ikeda wrote: On 2021-03-08 13:44, Fujii Masao wrote: On 2021/03/05 19:54, Masahiro Ikeda wrote: On 2021-03-05 12:47, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote

Re: [PATCH] pgbench: improve \sleep meta command

2021-03-08 Thread Fujii Masao
IIUC currently \sleep works in that way. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-09 Thread Fujii Masao
On 2021/03/09 4:47, David G. Johnston wrote: On Mon, Mar 8, 2021 at 8:48 AM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: Thanks for updating the patch! I applied cosmetic changes to that. Patch attached. Barring any objection, I will commit this version. Read ov

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-09 Thread Fujii Masao
On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao wrote: On 2021/03/03 14:33, Masahiro Ikeda wrote: On 2021-02-24 16:14, Fujii Masao

Re: shared-memory based stats collector

2021-03-09 Thread Fujii Masao
On 2021/03/09 16:51, Kyotaro Horiguchi wrote: At Sat, 6 Mar 2021 00:32:07 +0900, Fujii Masao wrote in On 2021/03/05 17:18, Kyotaro Horiguchi wrote: At Thu, 21 Jan 2021 12:03:48 +0900 (JST), Kyotaro Horiguchi wrote in Commit 960869da08 (database statistics) conflicted with this

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-09 Thread Fujii Masao
patch soon. +1. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: shared-memory based stats collector

2021-03-09 Thread Fujii Masao
On 2021/03/10 12:10, Kyotaro Horiguchi wrote: At Tue, 9 Mar 2021 23:24:10 +0900, Fujii Masao wrote in On 2021/03/09 16:51, Kyotaro Horiguchi wrote: At Sat, 6 Mar 2021 00:32:07 +0900, Fujii Masao wrote in I don't think that we should treat non-zero exit condition as a crash, as b

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Fujii Masao
On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao

Re: shared-memory based stats collector

2021-03-10 Thread Fujii Masao
On 2021/03/10 17:51, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 15:20:43 +0900, Fujii Masao wrote in On 2021/03/10 12:10, Kyotaro Horiguchi wrote: Agreed. The code moved to the original place and added the crash handling code. And I added a phrase to the comment. +* Was

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Fujii Masao
and localised change. +1 to push this kind of change into v14!! I've run into a couple of users who have just commented that recursive fsync() code out! BTW, we can skip that recursive fsync() by disabling fsync GUC even without commenting out the code? Regards, -- Fujii Masao Advanced

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Fujii Masao
On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote

Re: shared-memory based stats collector

2021-03-10 Thread Fujii Masao
On 2021/03/11 13:42, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund wrote in Hi, Two minor nits: Thanks for the comments! On 2021-03-10 21:47:51 +0900, Fujii Masao wrote: +/* Shared memory area for archiver process */ +typedef struct PgArchData

Re: Disallow cancellation of waiting for synchronous replication

2021-03-11 Thread Fujii Masao
). This prevents the transaction from seeing any data that have not been replicated yet. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-11 Thread Fujii Masao
On 2021/03/11 21:29, Masahiro Ikeda wrote: On 2021-03-11 11:52, Fujii Masao wrote: On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote

Re: shared-memory based stats collector

2021-03-11 Thread Fujii Masao
On 2021/03/12 9:23, Kyotaro Horiguchi wrote: At Thu, 11 Mar 2021 15:33:52 +0900, Fujii Masao wrote in On 2021/03/11 13:42, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund wrote in Hi, Two minor nits: Thanks for the comments! On 2021-03-10 21:47:51 +0900

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-11 Thread Fujii Masao
On 2021/03/11 21:29, Masahiro Ikeda wrote: On 2021-03-11 11:52, Fujii Masao wrote: On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-11 Thread Fujii Masao
On 2021/03/11 23:33, Fujii Masao wrote: On 2021/03/11 21:29, Masahiro Ikeda wrote: On 2021-03-11 11:52, Fujii Masao wrote: On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote

Re: shared-memory based stats collector

2021-03-11 Thread Fujii Masao
On 2021/03/12 13:49, Kyotaro Horiguchi wrote: At Fri, 12 Mar 2021 10:38:00 +0900 (JST), Kyotaro Horiguchi wrote in At Fri, 12 Mar 2021 10:03:31 +0900, Fujii Masao wrote in I moved archiver from the current location to next to "walsenders" that is to be terminated along wit

Re: HotStandbyActive() issue in postgres

2021-03-12 Thread Fujii Masao
is understanding is right, ISTM that HotStandbyActive() doesn't need to return false after recovery finishes because there is no user of it. No? Or you're implementing something that uses HotStandbyActive(), so want it to return false after the recovery? Regards, -- Fujii Masao Adv

Re: shared-memory based stats collector

2021-03-12 Thread Fujii Masao
On 2021/03/12 17:24, Kyotaro Horiguchi wrote: At Fri, 12 Mar 2021 15:13:15 +0900, Fujii Masao wrote in On 2021/03/12 13:49, Kyotaro Horiguchi wrote: I noticed that I accidentally removed the launch-suppression feature that is to avoid frequent relaunching. That mechanism is needed on the

Re: A new function to wait for the backend exit after termination

2021-03-14 Thread Fujii Masao
case? Isn't it better to throw ERROR like pg_promote() does? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Fujii Masao
for new WAL to be flushed by other processes during that period, so I think that it's better to use IPC as the type of the wait event WalSenderWaitForWAL. Thought? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Fujii Masao
On 2021/03/16 11:59, Kyotaro Horiguchi wrote: At Tue, 16 Mar 2021 03:12:54 +0900, Fujii Masao wrote in The wait event WalReceiverWaitStart has been categorized in the type Client. But why? Walreceiver is waiting for startup process to set the lsn and timeline while it is reporting

Re: fdatasync performance problem with large number of DB files

2021-03-16 Thread Fujii Masao
bit misleading? This may cause users to misunderstand that such fsync can happen only in the case of crash recovery. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: fdatasync performance problem with large number of DB files

2021-03-16 Thread Fujii Masao
nc would not be performed. This procedure is tricky. So IMO supporting sync_after_crash=none would be helpful for this case and simple. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: HotStandbyActive() issue in postgres

2021-03-16 Thread Fujii Masao
ndary and hot standby is active because you can connect to the server and UDF can be called. If hot standby is not active during recovery, you cannot conect to the server and cannot run UDF, so UDF doesn't need to handle the case where hot standby is not active during recovery. Thought? Regards,

Re: comment fix in postmaster.c

2021-03-16 Thread Fujii Masao
push the patch. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] pgbench: improve \sleep meta command

2021-03-16 Thread Fujii Masao
On 2021/03/09 0:54, Fujii Masao wrote: On 2021/03/08 23:10, Alvaro Herrera wrote: On 2021-Mar-08, kuroda.hay...@fujitsu.com wrote: Dear Fujii-san, Miyake-san Isn't it better to accept even negative sleep time like currently pgbench does? Otherwise we always need to check the variab

Re: Global snapshots

2020-10-28 Thread Fujii Masao
On 2020/10/23 11:58, Masahiko Sawada wrote: On Thu, 15 Oct 2020 at 01:41, Fujii Masao wrote: On 2020/09/17 15:56, Amit Kapila wrote: On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao wrote: One alternative is to add only hooks into PostgreSQL core so that we can implement the global

Re: [PATCH] Add features to pg_stat_statements

2020-10-28 Thread Fujii Masao
On 2020/10/28 17:31, seinoyu wrote: 2020-10-22 01:31 に Fujii Masao さんは書きました: On 2020/10/12 21:18, Yuki Seino wrote: The following review has been posted through the commitfest application: make installcheck-world:  tested, passed Implements feature:   tested, passed Spec compliant

Re: Allow some recovery parameters to be changed with reload

2020-10-28 Thread Fujii Masao
, archive recovery fails at the beginning. With the patch, how should we treat the case where retore_command is reset to empty during archive recovery? We should reject that change of restore_command? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Fujii Masao
mote" would be good reference to you. ERROR: function pg_terminate_backend(integer, boolean) does not exist at character 8 Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Add Information during standby recovery conflicts

2020-10-28 Thread Fujii Masao
ssage might not need to be output if the conflict is canceled due to max_standby_xxx_delay parameter. The latter message would be useful to know how long the recovery was waiting for the conflict. Thought? It's ok to implement this as a separate patch later, though. +Controls whether a log messag

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Fujii Masao
whole the database. BTW, with the patch, I observed that PREPARE TRANSACTION and COMMIT PREPARED caused assertion failure in my env, as I pointed upthread. How does the patch handle other feature depending on the existence of WAL, e.g., pg_logical_emit_message()? Regards, -- Fujii Masao Advanced C

Re: document pg_settings view doesn't display custom options

2020-10-28 Thread Fujii Masao
ettings displays pg_stat_statements.max after pg_stat_statements is loaded. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-10-28 Thread Fujii Masao
On 2020/10/07 11:30, Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 11:41 AM Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 11:20 AM Fujii Masao wrote: +1 Or it's also ok to make each patch separately. Anyway, thanks! Thanks. +1 to have separate patches. I will post two sep

Re: Disable WAL logging to speed up data loading

2020-10-29 Thread Fujii Masao
On 2020/10/29 19:21, Laurenz Albe wrote: On Thu, 2020-10-29 at 11:42 +0900, Fujii Masao wrote: But what if someone sets wal_level=none, performs some data modifications, sets wal_level=archive and after dome more processing decides to restore from a backup that was taken before the cluster

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-10-29 Thread Fujii Masao
easily extend pg_stat_wal so that any fields can be reported per process type. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-10-29 Thread Fujii Masao
ronous_commit" is useful for the workload. I just wonder how useful these counters are. Even without these counters, we already know synchronous_commit=off is likely to cause the better performance (but has the risk of data loss). So ISTM that these counters are not so useful when tuning

Re: Add Information during standby recovery conflicts

2020-10-29 Thread Fujii Masao
On 2020/10/30 10:29, Masahiko Sawada wrote: , On Thu, 29 Oct 2020 at 00:16, Fujii Masao wrote: On 2020/10/27 9:41, Masahiko Sawada wrote: On Tue, 20 Oct 2020 at 22:02, Drouvot, Bertrand wrote: Hi, On 10/15/20 9:15 AM, Masahiko Sawada wrote: CAUTION: This email originated from

Re: document pg_settings view doesn't display custom options

2020-10-29 Thread Fujii Masao
On 2020/10/29 21:54, John Naylor wrote: On Wed, Oct 28, 2020 at 11:38 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: On 2020/10/29 3:45, John Naylor wrote: > On Wed, Oct 28, 2020 at 2:15 PM John Naylor mailto:john.nay...@enterprisedb.com> <

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-11-02 Thread Fujii Masao
gards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-04 Thread Fujii Masao
On 2020/10/29 15:21, Fujii Masao wrote: On 2020/10/07 11:30, Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 11:41 AM Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 11:20 AM Fujii Masao wrote: +1 Or it's also ok to make each patch separately. Anyway, thanks! Thanks. +1 to

REFRESH MATERIALIZED VIEW and completion tag output

2020-11-05 Thread Fujii Masao
r to make REFRESH MATERIALIZED VIEW do the similar thing? Or we should not do that, for example, since changing the completion tag output may break the client app? [1] https://postgr.es/m/71f6bc72f8bbaa06e701f8bd2562c...@oss.nttdata.com Regards, -- Fujii Masao Advanced Computing Technology C

Re: REFRESH MATERIALIZED VIEW and completion tag output

2020-11-05 Thread Fujii Masao
On 2020/11/06 1:56, Mark Dilger wrote: On Nov 5, 2020, at 8:20 AM, Fujii Masao wrote: The patch that makes pg_stat_statements track the number of rows that REFRESH MATERIALIZED VIEW command processes was proposed at [1]. When reviewing the patch, I just wondered why the completion tag

Re: document pg_settings view doesn't display custom options

2020-11-06 Thread Fujii Masao
ain the next time somebody wants to add a para here. Okay, how's this? Looks good to me. Barring any objection, I will commit the patch. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Allow some recovery parameters to be changed with reload

2020-11-06 Thread Fujii Masao
(). Otherwise, when users forget to specify restore_command when starting archive recovery, recovery could wrongly proceed and the database could get corrupted. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-06 Thread Fujii Masao
On 2020/11/05 12:12, Kyotaro Horiguchi wrote: At Wed, 4 Nov 2020 21:16:29 +0530, Bharath Rupireddy wrote in On Wed, Nov 4, 2020 at 2:36 PM Fujii Masao wrote: Regarding other two patches, I think that it's better to use MyLatch rather than MyProc->procLatch or walrcv->latch i

Re: document pg_settings view doesn't display custom options

2020-11-08 Thread Fujii Masao
On 2020/11/07 0:42, Fujii Masao wrote: On 2020/10/31 2:06, John Naylor wrote: On Fri, Oct 30, 2020 at 12:48 PM Tom Lane mailto:t...@sss.pgh.pa.us>> wrote:     John Naylor mailto:john.nay...@enterprisedb.com>> writes: > Okay, along those lines here's a patch us

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-11-10 Thread Fujii Masao
On 2020/11/05 23:54, Seino Yuki wrote: 2020-11-02 20:01 に Fujii Masao さんは書きました: On 2020/11/02 14:02, Yuki Seino wrote: The following review has been posted through the commitfest application: make installcheck-world:  tested, passed Implements feature:   tested, passed Spec compliant

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-10 Thread Fujii Masao
dlers. Currently I've not found any actual issues by making walreceiver use standard SIGHUP handler, yet. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION diff --git a/src/backend/replication/walreceiver.c b/src/backend/r

Re: [PATCH] Add features to pg_stat_statements

2020-11-10 Thread Fujii Masao
nt because pg_stat_statements_info view is enough and there seems no use case for the function? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Allow some recovery parameters to be changed with reload

2020-11-11 Thread Fujii Masao
ommand to PGC_SIGHUP. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: recovery_target immediate timestamp

2020-11-11 Thread Fujii Masao
kpointstr), "%c", localtime(&time_tmp)); +#else + pg_strftime(checkpointstr, sizeof(checkpointstr), "%c", pg_localtime(&time_tmp, log_timezone)); +#endif You can simplify the code by using timestamptz_to_str() here instead, like xact_desc_commit() does.

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-11-11 Thread Fujii Masao
On 2020/11/10 17:29, Fujii Masao wrote: On 2020/11/05 23:54, Seino Yuki wrote: 2020-11-02 20:01 に Fujii Masao さんは書きました: On 2020/11/02 14:02, Yuki Seino wrote: The following review has been posted through the commitfest application: make installcheck-world:  tested, passed Implements

Re: Delay of standby shutdown

2020-11-11 Thread Fujii Masao
On 2020/11/04 9:35, Soumyadeep Chakraborty wrote: Hello Fujii, On Thu, Sep 17, 2020 at 6:49 AM Fujii Masao wrote: As far as I understand after debugging, the problem is as follows: Yes. 1. After the primary is stopped, and the standby startup process is waiting inside: (void

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-11 Thread Fujii Masao
On 2020/11/10 21:30, Bharath Rupireddy wrote: On Tue, Nov 10, 2020 at 3:04 PM Fujii Masao wrote: The main reason for having SetLatch() in SignalHandlerForConfigReload() is to wake up the calling process if waiting in WaitLatchOrSocket() or WaitLatch() and reload the new config file and

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-11-11 Thread Fujii Masao
On 2020/11/06 10:25, Masahiro Ikeda wrote: On 2020-10-30 11:50, Fujii Masao wrote: On 2020/10/29 17:03, Masahiro Ikeda wrote: Hi, Thanks for your comments and advice. I updated the patch. On 2020-10-21 18:03, Kyotaro Horiguchi wrote: At Tue, 20 Oct 2020 16:11:29 +0900, Masahiro Ikeda

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-11-11 Thread Fujii Masao
On 2020/11/12 14:58, Fujii Masao wrote: On 2020/11/06 10:25, Masahiro Ikeda wrote: On 2020-10-30 11:50, Fujii Masao wrote: On 2020/10/29 17:03, Masahiro Ikeda wrote: Hi, Thanks for your comments and advice. I updated the patch. On 2020-10-21 18:03, Kyotaro Horiguchi wrote: At Tue, 20

Re: recovery_target immediate timestamp

2020-11-13 Thread Fujii Masao
On 2020/11/13 8:39, Euler Taveira wrote: On Wed, 11 Nov 2020 at 22:40, Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: On 2020/11/12 6:00, Euler Taveira wrote: > The first patch adds a new message that prints the latest completed checkpoint > when th

Re: Delay of standby shutdown

2020-11-16 Thread Fujii Masao
On 2020/11/13 2:58, Soumyadeep Chakraborty wrote: Thanks! Marking this as ready for committer. Pushed. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] Add features to pg_stat_statements

2020-11-16 Thread Fujii Masao
actored the code and applied some cosmetic changes into the patch. Attached is the updated version of the patch that implements only "dealloc" part. Could you review this version? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT D

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-11-16 Thread Fujii Masao
On 2020/11/16 16:35, Masahiro Ikeda wrote: On 2020-11-12 14:58, Fujii Masao wrote: On 2020/11/06 10:25, Masahiro Ikeda wrote: On 2020-10-30 11:50, Fujii Masao wrote: On 2020/10/29 17:03, Masahiro Ikeda wrote: Hi, Thanks for your comments and advice. I updated the patch. On 2020-10-21 18

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-11-16 Thread Fujii Masao
written to the disk because WAL buffers got full  when to initialize a new WAL page' Or what about the following? Total number of times WAL data was written to the disk, to claim the buffer page to insert new WAL data when the WAL buffers got filled up with unwritten WAL data. R

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-17 Thread Fujii Masao
On 2020/11/13 20:24, Bharath Rupireddy wrote: On Thu, Nov 12, 2020 at 10:06 AM Fujii Masao wrote: Thanks for the analysis! I pushed the patch. Thanks! Since we are replacing custom SIGHUP and SIGTERM handlers with standard ones, how about doing the same thing in worker_spi.c? I posted a

Re: Add Information during standby recovery conflicts

2020-11-17 Thread Fujii Masao
timeouts[cnt].delay_ms = DeadlockTimeout; + cnt++; + } This needs to be executed only when the message has not been logged yet. Right? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [doc] plan invalidation when statistics are update

2020-11-17 Thread Fujii Masao
unds vague? Does the statement is re-analyzed and re-planned only when the planner statistics of database objects used in the statement are updated? If yes, we should describe that to make the note a bit more explicitly? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-18 Thread Fujii Masao
e what the right choice even is. I'd welcome some opinions. We can not use quickdie() here because as a bg worker we don't have to/can not send anything to client. We are good with SignalHandlerForCrashExit() as with all other bg workers. Thoughts? I think you're right. R

Re: bad logging around broken restore_command

2020-11-19 Thread Fujii Masao
committed. Thanks for the review! Pushed. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: walsender bug: stuck during shutdown

2020-11-23 Thread Fujii Masao
Dives reported the issue to? Sorry I could not find that.. I'd like to see the procedure to reproduce the issue. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-24 Thread Fujii Masao
heapScan->rs_startblock + ) % heapScan->rs_nblocks + 1); + prev_cblock = heapScan->rs_cblock; + } Regards, -- Fujii Masao Advanced Computing Technolo

Re: [doc] plan invalidation when statistics are update

2020-11-24 Thread Fujii Masao
On 2020/11/19 14:33, torikoshia wrote: On 2020-11-18 11:35, Fujii Masao wrote: Thanks for your comment! On 2020/11/18 11:04, torikoshia wrote: Hi, AFAIU, when the planner statistics are updated, generic plans are invalidated and PostgreSQL recreates. However, the manual doesn't se

Re: [PATCH] Add features to pg_stat_statements

2020-11-24 Thread Fujii Masao
On 2020/11/25 12:02, Seino Yuki wrote: 2020-11-17 01:46 に Fujii Masao さんは書きました: On 2020/11/16 12:22, Seino Yuki wrote: Thanks for updating the patch! +    pgss_info->dealloc = 0; +    SpinLockInit(&pgss_info->mutex); +    Assert(pgss_info->dealloc == 0); Why is t

Re: [doc] plan invalidation when statistics are update

2020-11-24 Thread Fujii Masao
On 2020/11/24 23:14, Fujii Masao wrote: On 2020/11/19 14:33, torikoshia wrote: On 2020-11-18 11:35, Fujii Masao wrote: Thanks for your comment! On 2020/11/18 11:04, torikoshia wrote: Hi, AFAIU, when the planner statistics are updated, generic plans are invalidated and PostgreSQL

Re: A few new options for CHECKPOINT

2020-11-24 Thread Fujii Masao
x27;s not convenient. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: walsender bug: stuck during shutdown

2020-11-24 Thread Fujii Masao
On 2020/11/25 0:07, Alvaro Herrera wrote: Hello, On 2020-Nov-24, Fujii Masao wrote: Thanks for working on this! Could you tell me the discussion thread where Chloe Dives reported the issue to? Sorry I could not find that.. It was not public -- sorry I didn't make that clear. I&#

Re: [PATCH] Add features to pg_stat_statements

2020-11-25 Thread Fujii Masao
On 2020/11/25 15:40, Seino Yuki wrote: 2020-11-25 13:13 に Fujii Masao さんは書きました: On 2020/11/25 12:02, Seino Yuki wrote: 2020-11-17 01:46 に Fujii Masao さんは書きました: On 2020/11/16 12:22, Seino Yuki wrote: Thanks for updating the patch! +    pgss_info->dealloc = 0; +    SpinLockI

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-25 Thread Fujii Masao
On 2020/11/25 0:25, Matthias van de Meent wrote: On Tue, 24 Nov 2020 at 15:05, Fujii Masao wrote: On 2020/11/21 2:32, Matthias van de Meent wrote: Hi, The pg_stat_progress_cluster view can report incorrect heap_blks_scanned values when synchronize_seqscans is enabled, because it allows

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-25 Thread Fujii Masao
On 2020/11/20 19:33, Bharath Rupireddy wrote: On Wed, Nov 18, 2020 at 5:52 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: > > > handle_sigterm() and die() are similar except that die() has extra > > handling(below) for exiting immediately when wait

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-11-25 Thread Fujii Masao
On 2020/11/19 16:31, Masahiro Ikeda wrote: On 2020-11-17 11:46, Fujii Masao wrote: On 2020/11/16 16:35, Masahiro Ikeda wrote: On 2020-11-12 14:58, Fujii Masao wrote: On 2020/11/06 10:25, Masahiro Ikeda wrote: On 2020-10-30 11:50, Fujii Masao wrote: On 2020/10/29 17:03, Masahiro Ikeda

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-25 Thread Fujii Masao
On 2020/11/25 20:52, Matthias van de Meent wrote: On Wed, 25 Nov 2020 at 10:35, Fujii Masao wrote: On 2020/11/25 0:25, Matthias van de Meent wrote: I noticed that with my proposed patch it is still possible to go to the next phase while heap_blks_scanned != heap_blks_total. This can

Re: walsender bug: stuck during shutdown

2020-11-25 Thread Fujii Masao
On 2020/11/26 3:45, Alvaro Herrera wrote: On 2020-Nov-25, Fujii Masao wrote: But whether MyWalSnd->write is InvalidRecPtr or not, if it's behind sentPtr, walsender should keep waiting for the ack to all the sent message to be replied, i.e., isn't this expected behavior of no

Re: [doc] plan invalidation when statistics are update

2020-11-25 Thread Fujii Masao
On 2020/11/26 14:30, torikoshia wrote: On 2020-11-25 14:13, Fujii Masao wrote: On 2020/11/24 23:14, Fujii Masao wrote: On 2020/11/19 14:33, torikoshia wrote: On 2020-11-18 11:35, Fujii Masao wrote: Thanks for your comment! On 2020/11/18 11:04, torikoshia wrote: Hi, AFAIU, when the

Re: walsender bug: stuck during shutdown

2020-11-25 Thread Fujii Masao
On 2020/11/26 11:45, Alvaro Herrera wrote: On 2020-Nov-26, Fujii Masao wrote: On the second thought, walsender doesn't wait forever unless wal_sender_timeout is disabled, even in the case in discussion? Or if there is the case where wal_sender_timeout doesn't work expectedly, we

Re: [PATCH] Add features to pg_stat_statements

2020-11-26 Thread Fujii Masao
On 2020/11/25 17:07, Fujii Masao wrote: On 2020/11/25 15:40, Seino Yuki wrote: 2020-11-25 13:13 に Fujii Masao さんは書きました: On 2020/11/25 12:02, Seino Yuki wrote: 2020-11-17 01:46 に Fujii Masao さんは書きました: On 2020/11/16 12:22, Seino Yuki wrote: Thanks for updating the patch

Re: Allow some recovery parameters to be changed with reload

2020-11-26 Thread Fujii Masao
can be reset to an empty) after archive recovery has started. But this updated version of description would be rather confusing to users. So I'm now thinking not to update that. Does anyone object to the patch? If no, I'm thinking to commit the patch. Regards, -- Fujii Masao Advanced

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-11-26 Thread Fujii Masao
On 2020/11/25 23:38, Bharath Rupireddy wrote: On Wed, Nov 25, 2020 at 3:29 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: > > On 2020/11/20 19:33, Bharath Rupireddy wrote: > > On Wed, Nov 18, 2020 at 5:52 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com

<    1   2   3   4   5   6   7   8   9   10   >