Recheck if ANALYZE is needed after VACUUM finishes by autovacuum

2025-02-04 Thread Masahiro Ikeda
the latest parameter values be referenced? The current implementation uses the parameter values at the time of VACUUM execution, except for pg_class.reltuples. I welcome your feedback. I’m open to alternative ideas and not stick to this particular approach. Regards, -- Masahiro Ikeda NTT

Re: Doc: fix the rewrite condition when executing ALTER TABLE ADD COLUMN

2025-01-06 Thread Masahiro Ikeda
On 2025-01-07 06:27, Robert Treat wrote: On Mon, Jan 6, 2025 at 3:18 AM Masahiro Ikeda wrote: On 2025-01-03 01:25, Robert Treat wrote: > On Tue, Dec 3, 2024 at 3:13 AM Masahiro Ikeda > wrote: >> >> Hi, >> >> The documentation seems to overlook the rewrite con

Re: Doc: fix the rewrite condition when executing ALTER TABLE ADD COLUMN

2025-01-06 Thread Masahiro Ikeda
On 2025-01-03 01:25, Robert Treat wrote: On Tue, Dec 3, 2024 at 3:13 AM Masahiro Ikeda wrote: Hi, The documentation seems to overlook the rewrite condition when executing ALTER TABLE ADD COLUMN. The current document states that a volatile DEFAULT will trigger a rewrite of the table and its

Re: Doc: clarify the log message level of the VERBOSE option

2025-01-05 Thread Masahiro Ikeda
On 2024-12-20 02:57, Fujii Masao wrote: On 2024/12/05 16:48, Masahiro Ikeda wrote: On 2024-12-05 16:23, Yugo NAGATA wrote: -  Prints a progress report as each table is clustered. +  Prints a progress report as each table is clustered, +  at INFO level. I feel like the comma

Re: Track the amount of time waiting due to cost_delay

2024-12-09 Thread Masahiro Ikeda
looks as simple as v9-0002 attached (0001 is same as for v8), so I think we should provide it. Thanks! The patch looks good to me. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Track the amount of time waiting due to cost_delay

2024-12-05 Thread Masahiro Ikeda
ases. I didn't see any comments regarding this, so I wanted to confirm. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Doc: clarify the log message level of the VERBOSE option

2024-12-04 Thread Masahiro Ikeda
to wait for a review from a native English speaker. Thanks. OK, I'll wait. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Doc: clarify the log message level of the VERBOSE option

2024-12-04 Thread Masahiro Ikeda
similar, I unified the descriptions for VACUUM, ANALYZE, and REINDEX. However, I don’t have a strong opinion on this. I’ve fixed it in the v2 patch. Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom f45982c314bda79904cbdcfee962040a1d30ae7e Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date:

Doc: clarify the log message level of the VERBOSE option

2024-12-03 Thread Masahiro Ikeda
BOSE option for EXPLAIN is not related to the log message level, and COPY already specifies the log message level. Therefore, I'd like to add descriptions for the VERBOSE option in the ANALYZE, VACUUM, and REINDEX commands. What do you think? Regards, -- Masahiro Ikeda NTT

Doc: fix the rewrite condition when executing ALTER TABLE ADD COLUMN

2024-12-03 Thread Masahiro Ikeda
added, or when a column with a domain data type that has constraints is added. What do you think? Regards, -- Masahiro Ikeda NTT DATA CORPORATION-- psql -f test.sql -e --no-psqlrc DROP TABLE IF EXISTS test; CREATE TABLE test (id1 int); INSERT INTO test (SELECT generate_series(1,100)); DELETE FROM

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2024-11-28 Thread Masahiro Ikeda
On 2024-11-26 07:32, Peter Geoghegan wrote: On Mon, Nov 25, 2024 at 4:07 AM Masahiro Ikeda wrote: One thing I am concerned about is that it reduces the cases where the optimization of _bt_checkkeys_look_ahead() works effectively, as the approach skips the skip immediately on the first

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2024-11-25 Thread Masahiro Ikeda
On 2024-11-23 07:34, Peter Geoghegan wrote: On Fri, Nov 22, 2024 at 4:17 AM Masahiro Ikeda wrote: Though the change fixes the assertion error in 'make check', there are still cases where the number of return values is incorrect. I would also like to continue investigating.

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2024-11-22 Thread Masahiro Ikeda
On 2024-11-21 17:47, Masahiro Ikeda wrote: On 2024-11-21 04:40, Peter Geoghegan wrote: diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c index b70b58e0c..ddae5f2a1 100644 --- a/src/backend/access/nbtree/nbtutils.c +++ b/src/backend/access/nbtree/nbtutils.c

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2024-11-21 Thread Masahiro Ikeda
On 2024-11-21 04:40, Peter Geoghegan wrote: On Wed, Nov 20, 2024 at 4:04 AM Masahiro Ikeda wrote: Thanks for your quick response! Attached is v16. This is similar to v15, but the new v16-0003-Fix-regressions* patch to fix the regressions is much less buggy, and easier to understand. Unlike

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2024-11-20 Thread Masahiro Ikeda
Hi Peter, On 2024-11-20 04:06, Peter Geoghegan wrote: Hi Masahiro, On Tue, Nov 19, 2024 at 3:30 AM Masahiro Ikeda wrote: Apologies for the delayed response. I've confirmed that the costing is significantly improved for multicolumn indexes in the case I provided. Thanks!

Re: Allow default \watch interval in psql to be configured

2024-11-19 Thread Masahiro Ikeda
interval in seconds that \watch waits between executions. The default value is 2.0. (5) I think it's better to replace queries with executions because the \watch documentation says so. + HELP0(" WATCH_INTERVAL\n" + "number of seconds \\watch waits between queries\n"); Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2024-11-19 Thread Masahiro Ikeda
TE INDEX test_idx on TEST (id1 int, id2 int, id3 no_eq_op, id4 int); * query: SELECT * FROM test WHERE id4 = 10; Regards, -- Masahiro Ikeda NTT DATA CORPORATIONDROP TABLE IF EXISTS t; CREATE TABLE t (id1 int, id2 int); -- 5x fast -- INSERT INTO t (SELECT 1, i%101 FROM generate_series(1,1_000_0

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-12 Thread Masahiro Ikeda
On 2024-11-13 00:55, Peter Geoghegan wrote: On Sun, Nov 10, 2024 at 11:36 PM Masahiro Ikeda wrote: Thanks! The change made it easier for me to understand. As follow-up to all of the recent work in this area, I'd like to add this wrapper function to return the next item from so->currP

Re: Fix to increment the index scan counter for the bloom filter index

2024-11-12 Thread Masahiro Ikeda
On 2024-11-13 06:23, Peter Geoghegan wrote: On Tue, Nov 12, 2024 at 5:01 AM Masahiro Ikeda wrote: I noticed that the bloom filter index forgets to increment the index scan counter while reviewing the skip scan patch [1]. It seems this was simply overlooked in the implementation. What do you

Fix to increment the index scan counter for the bloom filter index

2024-11-12 Thread Masahiro Ikeda
| 2024-11-12 18:15:39.270747+09 # idx_tup_read | 2503# idx_tup_fetch | 0 [1] https://www.postgresql.org/message-id/flat/CAH2-Wz%3DM_8UCPpD5GQuJXmQZ6xePwhVSss38TmkBAwic12VvCg%40mail.gmail.com#ffef7ecf393d0008c5ded2d2184a71f9 Regards, -- Masahiro Ikeda NTT

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-10 Thread Masahiro Ikeda
On 2024-11-11 12:19, Peter Geoghegan wrote: On Sun, Nov 10, 2024 at 9:53 PM Masahiro Ikeda wrote: I understand, thanks to your explanation. Cool. Now, there is a case where _bt_readnextpage() calls _bt_parallel_seize(), _bt_readpage() sets so->needPrimScan=true, and _bt_parallel_done()

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-10 Thread Masahiro Ikeda
On 2024-11-09 03:40, Peter Geoghegan wrote: On Fri, Nov 8, 2024 at 8:25 AM Masahiro Ikeda wrote: Since I couldn't understand the reason, I have a question: is the following deletion related to this change? @@ -770,7 +785,7 @@ _bt_parallel_done(IndexScanDesc

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-08 Thread Masahiro Ikeda
primitive index scan (defensive) +* primitive index scan */ Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-07 Thread Masahiro Ikeda
all relpages are being read to retrieve only 25% of the tuples. -- Without commit 1bd4bc8, the number was '236' in my environment. Planning Time: 0.105 ms Execution Time: 71.454 ms (18 rows) Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Doc: fix a description regarding WAL summarizer on glossary page

2024-06-13 Thread Masahiro Ikeda
s for applying. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Doc: fix a description regarding WAL summarizer on glossary page

2024-06-11 Thread Masahiro Ikeda
7;s one of backend processes. WAL summarizer (process) A special backend process that summarizes WAL data for incremental backups. https://www.postgresql.org/docs/devel/glossary.html Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Rethink the wait event names for postgres_fdw, dblink and etc

2023-10-05 Thread Masahiro Ikeda
of dblink were wrong for DblinkGetConnect: the wait event could be seen in other functions than dblink() and dblink_exec(). Thanks for modifying and committing. I agree your comments. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Make --help output fit within 80 columns per line

2023-08-22 Thread Masahiro Ikeda
Hi, On 2023-08-22 22:57, torikoshia wrote: On 2023-08-21 13:08, Masahiro Ikeda wrote: (2) Is there any reason that only src/bin commands are targeted? I found that we also need to fix vacuumlo with the above test. I think it's better to fix it because it's a contrib module.

Re: Fix pg_stat_reset_single_table_counters function

2023-08-20 Thread Masahiro Ikeda
me" in the collation tests so I have just used that, and applied the patch down to 15. Thanks! Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Fix pg_stat_reset_single_table_counters function

2023-08-20 Thread Masahiro Ikeda
On 2023-08-21 11:33, Kyotaro Horiguchi wrote: On 2023/08/15 14:13, Masahiro Ikeda wrote: On 2023-08-15 11:48, Masahiko Sawada wrote: +COMMENT ON DATABASE :current_database IS 'This is a test comment'; -- insert or update in 'pg_shdescription' I think the current_database s

Re: Make --help output fit within 80 columns per line

2023-08-20 Thread Masahiro Ikeda
c 'if [ `grep -e --help {} | wc -l` -gt 0 ]; then echo {}; fi' # programs which is tested with program_help_ok $ find -name "*.pl" | xargs -I {} sh -c 'grep -e program_help_ok {}' Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Rethink the wait event names for postgres_fdw, dblink and etc

2023-08-20 Thread Masahiro Ikeda
shorter like pgfdw_we_receive. * Update documents. * Add some tests with pg_wait_events view. Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 025525eae164e33117d94f2a90a4219808072b3c Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Mon, 21 Aug 2023 10:36:10 +0900 Subject: [PATCH] Make to use

Rethink the wait event names for postgres_fdw, dblink and etc

2023-08-17 Thread Masahiro Ikeda
ed. Feedback is always welcome and appreciated. Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 73c4c6562509465bea75a9bbd273298bdf0ee85e Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Fri, 18 Aug 2023 11:38:05 +0900 Subject: [PATCH] Make to use custom wait events for modules --- contrib/dblink/

Re: WIP: new system catalog pg_wait_event

2023-08-17 Thread Masahiro Ikeda
Hi, On 2023-08-17 14:53, Drouvot, Bertrand wrote: On 8/17/23 3:53 AM, Masahiro Ikeda wrote: 1) The regular expression needs to be changed in generate-wait_event_types.pl. I have compared the documentation with the output of the pg_wait_event view and found the following differences. * For

Re: Fix an entry in wait_event_names.txt

2023-08-16 Thread Masahiro Ikeda
patch to fix this entry in wait_event_names.txt (I did check that no other entries are in the same case). [1]: https://www.postgresql.org/message-id/735fbd560ae914c96faaa23cc8d9a118%40oss.nttdata.com Regards, +1. Thanks! Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: WIP: new system catalog pg_wait_event

2023-08-16 Thread Masahiro Ikeda
On 2023-08-17 10:57, Michael Paquier wrote: On Thu, Aug 17, 2023 at 10:53:02AM +0900, Masahiro Ikeda wrote: BTW, although I think this is outside the scope of this patch, it might be a good idea to be able to add a description to the API for custom wait events. Somebody on twitter has raised

Re: WIP: new system catalog pg_wait_event

2023-08-16 Thread Masahiro Ikeda
es */ 5) BTW, although I think this is outside the scope of this patch, it might be a good idea to be able to add a description to the API for custom wait events. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Fix pg_stat_reset_single_table_counters function

2023-08-14 Thread Masahiro Ikeda
On 2023-08-15 11:48, Masahiko Sawada wrote: On Mon, Aug 14, 2023 at 5:12 PM Masahiro Ikeda wrote: I changed the table to check the stats from pg_database to pg_shdescription because the stats can update via the SQL interface COMMENT command. It seems to work well. +COMMENT ON DATABASE

Re: Support to define custom wait events for extensions

2023-08-14 Thread Masahiro Ikeda
On 2023-08-14 18:26, Michael Paquier wrote: On Mon, Aug 14, 2023 at 05:55:42PM +0900, Masahiro Ikeda wrote: I'm thinking a name like "contrib name + description summary" would be nice. The "contrib name" is namespace-like and the "description summary" is

Re: Support to define custom wait events for extensions

2023-08-14 Thread Masahiro Ikeda
On 2023-08-14 15:28, Michael Paquier wrote: On Mon, Aug 14, 2023 at 12:31:05PM +0900, Masahiro Ikeda wrote: Thanks! I confirmed the changes, and all tests passed. Okay, cool. I got some extra time today and applied that, with a few more tweaks. Thanks for applying master branch! This

Re: Fix pg_stat_reset_single_table_counters function

2023-08-14 Thread Masahiro Ikeda
mmand. Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 215ef8ef68af30753cfcd4336b1f6bd9203ac014 Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Mon, 14 Aug 2023 16:48:30 +0900 Subject: [PATCH] Fix pg_stat_reset_single_table_counters function. This commit revives the feature to reset s

Re: Support to define custom wait events for extensions

2023-08-13 Thread Masahiro Ikeda
other in-core modules. Thanks! I confirmed the changes, and all tests passed. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Fix pg_stat_reset_single_table_counters function

2023-08-09 Thread Masahiro Ikeda
On 2023-08-01 15:23, Masahiro Ikeda wrote: Hi, My colleague, Mitsuru Hinata (in CC), found the following issue. The documentation of pg_stat_reset_single_table_counters() says pg_stat_reset_single_table_counters ( oid ) → void Resets statistics for a single table or index in the current

Re: Support to define custom wait events for extensions

2023-08-09 Thread Masahiro Ikeda
for closing the former entry. (https://commitfest.postgresql.org/43/4368/) Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 234cc7d852aebf78285ccde383f27ea35a27dad2 Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Thu, 10 Aug 2023 10:44:41 +0900 Subject: [PATCH 2/2] poc: custom wait event for

Re: Support to define custom wait events for extensions

2023-08-09 Thread Masahiro Ikeda
atch) I'm worrying about * Is 512(wee_hash_max_size) the maximum number of the custom wait events sufficient? * Is there any way to not force extensions that don't use shared memory for their use like dblink to acquire AddinShmemInitLock?; Regards, -- Masahiro Ikeda NTT

Re: Support to define custom wait events for extensions

2023-08-08 Thread Masahiro Ikeda
I accidentally attached a patch in one previous email. But, you don't need to check it, sorry. (v1-0001-Change-to-manage-custom-wait-events-in-shared-hash.patch) Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-08-08 Thread Masahiro Ikeda
On 2023-08-08 10:05, Michael Paquier wrote: On Tue, Aug 08, 2023 at 09:39:02AM +0900, Masahiro Ikeda wrote: I am thinking a bit that we also need another hash where the key is a custom string. For extensions that have no dependencies with shared_preload_libraries, I think we need to avoid that

Re: Support to define custom wait events for extensions

2023-08-07 Thread Masahiro Ikeda
On 2023-08-08 08:54, Michael Paquier wrote: On Wed, Aug 02, 2023 at 06:34:15PM +0900, Masahiro Ikeda wrote: On 2023-08-01 12:23, Andres Freund wrote: This is why the scheme as implemented doesn't really make sense to me. It'd be much easier to use if we had a shared hashtabl

Re: Support to define custom wait events for extensions

2023-08-02 Thread Masahiro Ikeda
On 2023-08-01 12:23, Andres Freund wrote: Hi, On 2023-08-01 12:14:49 +0900, Michael Paquier wrote: On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote: > Thanks for committing the main patch. > > In my understanding, the rest works are > * to support WaitEventExten

Re: Support to define custom wait events for extensions

2023-08-01 Thread Masahiro Ikeda
On 2023-08-02 08:38, Ranier Vilela wrote: Hi, On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote: Thanks for committing the main patch. Latest head Ubuntu 64 bits gcc 13 64 bits ./configure --without-icu make clean make In file included from ../../src/include/pgstat.h:20

Fix pg_stat_reset_single_table_counters function

2023-07-31 Thread Masahiro Ikeda
an -- 0 (1 row) Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 561c4391c9dac30b5478637a6baf8c8689226da5 Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Tue, 1 Aug 2023 13:46:00 +0900 Subject: [PATCH] Fix pg_stat_reset_single_table_counters function. This commit reviv

Re: Support to define custom wait events for extensions

2023-07-31 Thread Masahiro Ikeda
WAIT_EVENT_EXTENSION to custom wait events Do someone already works for them? If not, I'll consider how to realize them. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-07-31 Thread Masahiro Ikeda
On 2023-07-31 16:28, Michael Paquier wrote: On Mon, Jul 31, 2023 at 03:53:27PM +0900, Masahiro Ikeda wrote: /* This should only be called for user-defined wait event. */ if (eventId < NUM_BUILTIN_WAIT_EVENT_EXTENSION) ereport(ER

Re: Support to define custom wait events for extensions

2023-07-30 Thread Masahiro Ikeda
ndering if it should also check the eventId that has been allocated though it needs to take the spinlock and GetWaitEventExtensionIdentifier() doesn't take it into account. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-07-27 Thread Masahiro Ikeda
nextId; /* next ID to assign */ + slock_t mutex; /* protects the counter only */ +} WaitEventExtensionCounter; + +/* pointer to the shared memory */ +static WaitEventExtensionCounter * waitEventExtensionCounter; Regards, -

Re: Support worker_spi to execute the function dynamically.

2023-07-24 Thread Masahiro Ikeda
On 2023-07-24 12:01, Bharath Rupireddy wrote: I'm attaching the v3 patch. I verified it works and it looks good to me. Thanks to your work, I will be able to implement tests for custom wait events. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support worker_spi to execute the function dynamically.

2023-07-23 Thread Masahiro Ikeda
On 2023-07-22 01:05, Bharath Rupireddy wrote: On Fri, Jul 21, 2023 at 4:05 PM Masahiro Ikeda wrote: (2) Do we need "worker_spi.total_workers = 0" and "shared_preload_libraries = worker_spi" in dynamic.conf. Currently, the static bg workers will not

Re: Support worker_spi to execute the function dynamically.

2023-07-21 Thread Masahiro Ikeda
Hi, On 2023-07-20 18:39, Bharath Rupireddy wrote: On Thu, Jul 20, 2023 at 2:59 PM Michael Paquier wrote: On Thu, Jul 20, 2023 at 05:54:55PM +0900, Masahiro Ikeda wrote: > Yes, you're right. When I tried using worker_spi to test wait event, > I found the behavior. And thanks a l

Re: Support worker_spi to execute the function dynamically.

2023-07-20 Thread Masahiro Ikeda
master[853612] LOG: background worker "worker_spi" (PID 853621) exited with exit code 1 ``` It's better to remove "shared_preload_libraries = worker_spi" from the test configuration. I misunderstood that two background workers would be launched and waiting at the start

Re: Support worker_spi to execute the function dynamically.

2023-07-20 Thread Masahiro Ikeda
On 2023-07-20 12:55, Michael Paquier wrote: On Thu, Jul 20, 2023 at 11:15:51AM +0900, Masahiro Ikeda wrote: While I'm working on the thread[1], I found that the function of worker_spi module fails if 'shared_preload_libraries' doesn't have worker_spi. I guess that you were

Support worker_spi to execute the function dynamically.

2023-07-19 Thread Masahiro Ikeda
r extensions https://www.postgresql.org/message-id/flat/b9f5411acda0cf15c8fbb767702ff43e%40oss.nttdata.com Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom c6e60c66c4066b4a01981ffae5a168901e7283eb Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Thu, 20 Jul 2023 10:34:50 +0900 Subject: [PATCH]

Re: Support to define custom wait events for extensions

2023-07-18 Thread Masahiro Ikeda
On 2023-07-19 12:52, Masahiro Ikeda wrote: Hi, I updated the patches. * v6-0001-Support-custom-wait-events-for-extensions.patch I updated the patch since the cfbot found a bug. * v7-0001-Support-custom-wait-events-for-extensions.patch Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom

Re: Support to define custom wait events for extensions

2023-07-18 Thread Masahiro Ikeda
after the APIs are committed. The example mentioned in docs should be updated to the contrib modules codes, not the test module. Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 9c5c1f1f4f0be87cc06c7127396e7d228b665b8a Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Wed, 19 Jul 2023

Re: Support to define custom wait events for extensions

2023-07-12 Thread Masahiro Ikeda
On 2023-07-13 09:12, Michael Paquier wrote: On Wed, Jul 12, 2023 at 05:46:31PM +0900, Michael Paquier wrote: On Wed, Jul 12, 2023 at 04:52:38PM +0900, Masahiro Ikeda wrote: If the behavior is unexpected, we need to change the current code. I have created a patch for the areas that I felt

Re: Support to define custom wait events for extensions

2023-07-12 Thread Masahiro Ikeda
purpose. This is because the test will execute commands serially, while a function waiting is executed in a backend process, it is not possible to connect to another backend and check the wait events on pg_stat_activity view. Please let me know if my understanding is wrong. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-07-12 Thread Masahiro Ikeda
On 2023-07-12 02:39, Tristan Partin wrote: From bf06b8100cb747031959fe81a2d19baabc4838cf Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Fri, 16 Jun 2023 11:53:29 +0900 Subject: [PATCH 1/2] Support custom wait events for extensions. + * This is indexed by event ID minus

Re: Support to define custom wait events for extensions

2023-07-12 Thread Masahiro Ikeda
On 2023-07-11 16:45, Michael Paquier wrote: On Fri, Jun 23, 2023 at 05:56:26PM +0900, Masahiro Ikeda wrote: I updated the patches to handle the warning mentioned by PostgreSQL Patch Tester, and removed unnecessary spaces. I have begun hacking on that, and the API layer inspired from the

Re: PG 16 draft release notes ready

2023-06-30 Thread Masahiro Ikeda
feature is reverted with the commit. * https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=2535c74b1a6190cc42e13f6b6b55d94bff4b7dd6 Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-06-23 Thread Masahiro Ikeda
Hi, I updated the patches to handle the warning mentioned by PostgreSQL Patch Tester, and removed unnecessary spaces. Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 1bb78fa2cbe6b030cea7a570bec88bd4d68f314a Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Fri, 23 Jun 2023 17:38:38 +0900

Re: Support to define custom wait events for extensions

2023-06-21 Thread Masahiro Ikeda
On 2023-06-20 18:26, Masahiro Ikeda wrote: The followings are TODO items. * to check that meson.build works since I tested with old command `make` now I test with meson and I updated the patches to work with it. My test procedure is the following. ``` export builddir=/mnt/tmp/build export

Re: Support to define custom wait events for extensions

2023-06-20 Thread Masahiro Ikeda
wait events for existing contrib modules (ex. postgres_fdw) Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 206ce9e1d74d1449b50cfc765936172de98687e8 Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Tue, 20 Jun 2023 17:59:34 +0900 Subject: [PATCH 2/2] Add test codes for custom wait events

Re: Support to define custom wait events for extensions

2023-06-16 Thread Masahiro Ikeda
On 2023/06/17 1:16, Tristan Partin wrote: > I will take a look at your V2 when it is ready! I will also pass along > that this is wanted by Neon customers :). Thanks! Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-06-16 Thread Masahiro Ikeda
On 2023-06-16 16:46, Michael Paquier wrote: On Fri, Jun 16, 2023 at 11:14:05AM +0900, Masahiro Ikeda wrote: I tried to query on pg_stat_activity to check the background worker invoked by pg_prewarm. But, I found that pg_stat_activity doesn't show it although I may be missing something...

Re: Support to define custom wait events for extensions

2023-06-15 Thread Masahiro Ikeda
. As a Neon user, I also feel the feature is important. Same as you. Thanks to Michael and Drouvot, I got to know the word tranche and the related existing code. From 59a118402e5e59685fb9e0fb086872e25a405736 Mon Sep 17 00:00:00 2001 From: Masahiro Ikeda Date: Thu, 15 Jun 2023 12:57:29 +0900

Re: Support to define custom wait events for extensions

2023-06-15 Thread Masahiro Ikeda
On 2023-06-15 22:21, Drouvot, Bertrand wrote: Hi, On 6/15/23 10:00 AM, Michael Paquier wrote: On Thu, Jun 15, 2023 at 03:06:01PM +0900, Masahiro Ikeda wrote: Currently, only one PG_WAIT_EXTENSION event can be used as a wait event for extensions. Therefore, in environments with multiple

Re: Support to define custom wait events for extensions

2023-06-15 Thread Masahiro Ikeda
Thanks for replying and your kind advice! On 2023-06-15 17:00, Michael Paquier wrote: On Thu, Jun 15, 2023 at 03:06:01PM +0900, Masahiro Ikeda wrote: In the core, the requested wait events are dynamically registered in shared memory. The extension then obtains the wait event information with

Support to define custom wait events for extensions

2023-06-14 Thread Masahiro Ikeda
(ex. postgres_fdw) * add regression code (but, it seems to be difficult) * others? (Please let me know) [1] https://www.postgresql.org/message-id/81290a48-b25c-22a5-31a6-3feff5864fe3%40gmail.com Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom 59a118402e5e59685fb9e0fb086872e25a405736 Mon Se

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

2023-06-09 Thread Masahiro Ikeda
Hi, On 2023-06-09 13:26, Drouvot, Bertrand wrote: Hi, On 6/9/23 1:15 AM, Michael Paquier wrote: On Thu, Jun 08, 2023 at 10:57:55AM +0900, Masahiro Ikeda wrote: (Excuse me for cutting in, and this is not directly related to the thread.) +1. I'm interested in the feature. This is j

Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN

2023-06-07 Thread Masahiro Ikeda
nd it probable be useful for other users. IMO, at least, it's better to improve the specification that "Extension" wait event type has only the "Extension" wait event. [1] https://github.com/neondatabase/neon Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: LogicalChanges* and LogicalSubxact* wait events are never reported

2021-10-21 Thread Masahiro Ikeda
On 2021/10/21 17:40, Amit Kapila wrote: > On Wed, Oct 20, 2021 at 3:46 PM Masahiro Ikeda > wrote: >> >> On 2021/10/20 18:17, Amit Kapila wrote: >>> On Wed, Oct 20, 2021 at 10:50 AM Michael Paquier >>> wrote: >>>> >>>> On Wed, Oct

Re: LogicalChanges* and LogicalSubxact* wait events are never reported

2021-10-20 Thread Masahiro Ikeda
On 2021/10/20 18:17, Amit Kapila wrote: > On Wed, Oct 20, 2021 at 10:50 AM Michael Paquier wrote: >> >> On Wed, Oct 20, 2021 at 02:12:20PM +0900, Masahiro Ikeda wrote: >>> If my understanding is right, it's better to remove them since they make >>> users c

LogicalChanges* and LogicalSubxact* wait events are never reported

2021-10-19 Thread Masahiro Ikeda
e the attached patch. I confirmed that to make check-world passes all tests. [1] https://www.postgresql.org/message-id/CAA4eK1JV37jXUT5LeWzkBDNNnSntwQbLUZAj6m82QMiR1ZuuHQ%40mail.gmail.com Regards, -- Masahiro Ikeda NTT DATA CORPORATION diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring

Re: Allow escape in application_name

2021-09-06 Thread Masahiro Ikeda
On 2021/09/06 21:28, Fujii Masao wrote: > > > On 2021/09/06 16:48, Masahiro Ikeda wrote: >> On 2021-09-06 13:21, kuroda.hay...@fujitsu.com wrote: >>> Dear Ikeda-san, >>> >>>> I agree that this feature is useful. Thanks for working this. &g

Re: Allow escape in application_name

2021-09-06 Thread Masahiro Ikeda
te it. How do you think? OK. I agree that (1) is enough for the first step. If I have any time, I'll investigate it too. Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Allow escape in application_name

2021-09-05 Thread Masahiro Ikeda
ase the limitation of application_name and make it accept Non-ASCII. As a user, 3) is best for me. But it seems be out of scope this threads, so will you select 1)? Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: RFC: Logging plan of the running query

2021-07-13 Thread Masahiro Ikeda
ing query, not its planning phase. If so, the planning buffer usage is not so much useful. But, since the overhead to output buffer usages is not high and it's used for debugging use cases, I wonder it's not a bad idea to output buffer usages too. Thought? Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Fix comments of heap_prune_chain()

2021-07-13 Thread Masahiro Ikeda
client and I confirmed that I could send an email with a new patch. So, please check it. https://www.postgresql.org/message-id/1aa07e2a-b715-5649-6c62-4fff96304d18%40oss.nttdata.com Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Fix comments of heap_prune_chain()

2021-07-13 Thread Masahiro Ikeda
s a message instead. Thanks for your comments. I agree your suggestions. I also updated prstate->vistest to heap_prune_satisfies_vacuum of v1 patch because heap_prune_satisfies_vacuum() tests with not only prstate->vistest but also prstate->old_snap_xmin. I think it's more accu

Re: Transactions involving multiple postgres foreign servers, take 2

2021-07-04 Thread Masahiro Ikeda
On 2021/06/30 10:05, Masahiko Sawada wrote: > On Fri, Jun 25, 2021 at 9:53 AM Masahiro Ikeda > wrote: >> >> Hi Jamison-san, sawada-san, >> >> Thanks for testing! >> >> FWIF, I tested using pgbench with "--rate=" option to know the server &g

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiro Ikeda
: Distributed PostgreSQL for Data Intensive Applications >From 12:27 says that how to solve unresolved prepared xacts. https://www.youtube.com/watch?v=AlF4C60FdlQ&list=PL3xUNnH4TdbsfndCMn02BqAAgGB0z7cwq Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiro Ikeda
the > upper bound if committing foreign prepared transactions cannot keep > up. Hi Jamison-san, sawada-san, Thanks for testing! FWIF, I tested using pgbench with "--rate=" option to know the server can execute transactions with stable throughput. As sawada-san said, the latest patch resolved second phase of 2PC asynchronously. So, it's difficult to control the stable throughput without "--rate=" option. I also worried what I should do when the error happened because to increase "max_prepared_foreign_transaction" doesn't work. Since too overloading may show the error, is it better to add the case to the HINT message? BTW, if sawada-san already develop to run the resolver processes in parallel, why don't you measure performance improvement? Although Robert-san, Tunakawa-san and so on are discussing what architecture is best, one discussion point is that there is a performance risk if adopting asynchronous approach. If we have promising solutions, I think we can make the discussion forward. In my understanding, there are three improvement idea. First is that to make the resolver processes run in parallel. Second is that to send "COMMIT/ABORT PREPARED" remote servers in bulk. Third is to stop syncing the WAL remove_fdwxact() after resolving is done, which I addressed in the mail sent at June 3rd, 13:56. Since third idea is not yet discussed, there may be my misunderstanding. -- Masahiro Ikeda NTT DATA CORPORATION

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-02 Thread Masahiro Ikeda
On 2021/05/25 21:59, Masahiko Sawada wrote: > On Fri, May 21, 2021 at 5:48 PM Masahiro Ikeda > wrote: >> >> On 2021/05/21 13:45, Masahiko Sawada wrote: >>> >>> Yes. We also might need to be careful about the order of foreign >>> transaction r

Re: Transactions involving multiple postgres foreign servers, take 2

2021-05-21 Thread Masahiro Ikeda
On 2021/05/21 13:45, Masahiko Sawada wrote: > On Fri, May 21, 2021 at 12:45 PM Masahiro Ikeda > wrote: >> >> >> >> On 2021/05/21 10:39, Masahiko Sawada wrote: >>> On Thu, May 20, 2021 at 1:26 PM Masahiro Ikeda >>> wrote: >>>> >

Re: Transactions involving multiple postgres foreign servers, take 2

2021-05-20 Thread Masahiro Ikeda
On 2021/05/21 10:39, Masahiko Sawada wrote: > On Thu, May 20, 2021 at 1:26 PM Masahiro Ikeda > wrote: >> >> >> On 2021/05/11 13:37, Masahiko Sawada wrote: >>> I've attached the updated patches that incorporated comments from >>> Zhihong and I

Re: Transactions involving multiple postgres foreign servers, take 2

2021-05-19 Thread Masahiro Ikeda
on participant/ - s/Foreign transactions involved in the current transaction/Foreign transaction participants involved in the current transaction/ Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: wal stats questions

2021-05-19 Thread Masahiro Ikeda
On 2021/05/19 11:40, Fujii Masao wrote: > Thanks for updating the patch! I modified some comments slightly and > pushed that version of the patch. Thanks a lot! Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: wal stats questions

2021-05-17 Thread Masahiro Ikeda
On 2021/05/17 22:34, Fujii Masao wrote: > > > On 2021/05/17 16:39, Masahiro Ikeda wrote: >> >> Thanks for your comments! >> >>>> + * is executed, wal records aren't nomally generated (although HOT >>>> makes >>> >>>

Re: wal stats questions

2021-05-17 Thread Masahiro Ikeda
er may write/sync the WAL although it doesn't > > You use both 'wal' and 'WAL' in the comments, but are there any intension? No, I changed to use 'WAL' only. Although some other comments have 'wal' and 'WAL', it seems that 'WAL' is

Re: wal stats questions

2021-05-12 Thread Masahiro Ikeda
On 2021/05/12 19:19, Fujii Masao wrote: > > > On 2021/05/11 18:46, Masahiro Ikeda wrote: >> >> >> On 2021/05/11 16:44, Fujii Masao wrote: >>> >>> >>> On 2021/04/28 9:10, Masahiro Ikeda wrote: >>>> >>>> >>>&g

Re: wal stats questions

2021-05-11 Thread Masahiro Ikeda
On 2021/05/11 16:44, Fujii Masao wrote: > > > On 2021/04/28 9:10, Masahiro Ikeda wrote: >> >> >> On 2021/04/27 21:56, Fujii Masao wrote: >>> >>> >>> On 2021/04/26 10:11, Masahiro Ikeda wrote: >>>> >>>> Fir

  1   2   3   >