Re: Small and unaffected typo in pg_logical_slot_get_changes_guts()

2022-02-16 Thread Kasahara Tatsuhito
2022年2月17日(木) 10:56 Michael Paquier : > > On Wed, Feb 16, 2022 at 01:25:09PM +0900, Kasahara Tatsuhito wrote: > > Remove all references to tuplestore_donestoring() except for the header. > > Looks fine, thanks. This has no impact on Melanie's patch posted on > [1], so

Re: Small and unaffected typo in pg_logical_slot_get_changes_guts()

2022-02-15 Thread Kasahara Tatsuhito
Hi ! 2022年2月16日(水) 11:42 Michael Paquier : > > On Wed, Feb 16, 2022 at 11:27:58AM +0900, Kasahara Tatsuhito wrote: > > Since commit dd04e958c8b03c0f0512497651678c7816af3198, > > tuplestore_donestoring() seems to be left only for compatibility, so > > it looks like it can

Small and unaffected typo in pg_logical_slot_get_changes_guts()

2022-02-15 Thread Kasahara Tatsuhito
Hi, I noticed the small typo in pg_logical_slot_get_changes_guts(). == --- a/src/backend/replication/logical/logicalfuncs.c +++ b/src/backend/replication/logical/logicalfuncs.c @@ -295,7 +295,7 @@ pg_logical_slot_get_changes_guts(FunctionCallInfo fcinfo, bool c

Re: Error code for checksum failure in origin.c

2021-08-30 Thread Kasahara Tatsuhito
On Mon, Aug 30, 2021 at 5:35 PM Amit Kapila wrote: > > On Fri, Aug 27, 2021 at 12:47 PM Daniel Gustafsson wrote: > > > > > On 27 Aug 2021, at 06:32, Amit Kapila wrote: > > > > > I think we need to backpatch this till 9.6 as this is introduced by > > > commit 5aa2350426. Any objections to that? >

Re: Error code for checksum failure in origin.c

2021-08-26 Thread Kasahara Tatsuhito
On Fri, Aug 27, 2021 at 1:32 PM Amit Kapila wrote: > > On Thu, Aug 26, 2021 at 4:11 PM Daniel Gustafsson wrote: > > > > > On 26 Aug 2021, at 12:03, Amit Kapila wrote: > > > > > > On Thu, Aug 26, 2021 at 3:18 PM Kasahara Tatsuhito > > > wrote: &g

Error code for checksum failure in origin.c

2021-08-26 Thread Kasahara Tatsuhito
Hi. In the code in src/backend/replication/logical/origin.c, the error code "ERRCODE_CONFIGURATION_LIMIT_EXCEEDED" is given when a checksum check results in an error, but "ERRCODE_ DATA_CORRUPTED" seems to be more appropriate. diff --git a/src/backend/replication/logical/orig

Re: There doesn't seem to be any case where PQputCopyEnd() returns 0

2021-02-06 Thread Kasahara Tatsuhito
On Fri, Feb 5, 2021 at 11:01 PM Fujii Masao wrote: > > > > On 2021/02/05 16:52, Kasahara Tatsuhito wrote: > > Hi, > > > > The following is written in the comments of PQputCopyEnd(). > > > > (snip) > > * Returns 1 if successful, 0 if data coul

There doesn't seem to be any case where PQputCopyEnd() returns 0

2021-02-04 Thread Kasahara Tatsuhito
Hi, The following is written in the comments of PQputCopyEnd(). (snip) * Returns 1 if successful, 0 if data could not be sent (only possible * in nonblock mode), or -1 if an error occurs. (snip) The PQputCopyEnd() section of the manual (libpq.sgml) describes the following. The result

Re: Get memory contexts of an arbitrary backend process

2020-12-25 Thread Kasahara Tatsuhito
Hi, On Thu, Dec 10, 2020 at 10:48 AM torikoshia wrote: > > On 2020-12-04 19:16, torikoshia wrote: > > On 2020-12-03 10:36, Tom Lane wrote: > >> Fujii Masao writes: > >>> I'm starting to study how this feature behaves. At first, when I > >>> executed > >>> the following query, the function never

Re: autovac issue with large number of tables

2020-12-08 Thread Kasahara Tatsuhito
On Wed, Dec 9, 2020 at 12:01 AM Fujii Masao wrote: > > > > On 2020/12/04 12:21, Kasahara Tatsuhito wrote: > > Hi, > > > > On Thu, Dec 3, 2020 at 9:09 PM Fujii Masao > > wrote: > >> > >> > >> > >> On 2020/12/03 11:46, Kasaha

Re: autovac issue with large number of tables

2020-12-03 Thread Kasahara Tatsuhito
Hi, On Thu, Dec 3, 2020 at 9:09 PM Fujii Masao wrote: > > > > On 2020/12/03 11:46, Kasahara Tatsuhito wrote: > > On Wed, Dec 2, 2020 at 7:11 PM Masahiko Sawada > > wrote: > >> > >> On Wed, Dec 2, 2020 at 3:33 PM Fujii Masao > >> wrote: &g

Re: autovac issue with large number of tables

2020-12-02 Thread Kasahara Tatsuhito
> >> > > >> On Tue, Dec 1, 2020 at 4:32 PM Fujii Masao > > >> wrote: > > >>> > > >>> > > >>> > > >>> On 2020/12/01 16:23, Masahiko Sawada wrote: > > >>>> On Tue,

Re: autovac issue with large number of tables

2020-12-02 Thread Kasahara Tatsuhito
>> > >>> > >>> > >>> On 2020/12/01 16:23, Masahiko Sawada wrote: > >>>> On Tue, Dec 1, 2020 at 1:48 PM Kasahara Tatsuhito > >>>> wrote: > >>>>> > >>>>> Hi, > >>>>> > >

Re: autovac issue with large number of tables

2020-11-30 Thread Kasahara Tatsuhito
Hi, On Mon, Nov 30, 2020 at 8:59 PM Fujii Masao wrote: > > > > On 2020/11/30 10:43, Masahiko Sawada wrote: > > On Sun, Nov 29, 2020 at 10:34 PM Kasahara Tatsuhito > > wrote: > >> > >> Hi, Thanks for you comments. > >> > >&

Re: autovac issue with large number of tables

2020-11-29 Thread Kasahara Tatsuhito
Hi, Thanks for you comments. On Fri, Nov 27, 2020 at 9:51 PM Fujii Masao wrote: > > > > On 2020/11/27 18:38, Kasahara Tatsuhito wrote: > > Hi, > > > > On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao > > wrote: > >> > >> > >> > >

Re: autovac issue with large number of tables

2020-11-27 Thread Kasahara Tatsuhito
On Fri, Nov 27, 2020 at 5:22 PM Masahiko Sawada wrote: > > On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao > wrote: > > > > > > > > On 2020/11/26 10:41, Kasahara Tatsuhito wrote: > > > On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada > > > w

Re: autovac issue with large number of tables

2020-11-27 Thread Kasahara Tatsuhito
Hi, On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao wrote: > > > > On 2020/11/26 10:41, Kasahara Tatsuhito wrote: > > On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada > > wrote: > >> > >> On Wed, Nov 25, 2020 at 4:18 PM Kasahara Tatsuhito > >>

Re: autovac issue with large number of tables

2020-11-25 Thread Kasahara Tatsuhito
On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada wrote: > > On Wed, Nov 25, 2020 at 4:18 PM Kasahara Tatsuhito > wrote: > > > > Hi, > > > > On Wed, Nov 25, 2020 at 2:17 PM Masahiko Sawada > > wrote: > > > > > > On Fri, Sep 4, 2020 at 7:

Re: autovac issue with large number of tables

2020-11-24 Thread Kasahara Tatsuhito
Hi, On Wed, Nov 25, 2020 at 2:17 PM Masahiko Sawada wrote: > > On Fri, Sep 4, 2020 at 7:50 PM Kasahara Tatsuhito > wrote: > > > > Hi, > > > > On Wed, Sep 2, 2020 at 2:10 AM Kasahara Tatsuhito > > wrote: > > > > I wonder if we could have table_

Re: Add a description to the documentation that toast_tuple_target affects "Main"

2020-10-12 Thread Kasahara Tatsuhito
Hi, On Fri, Oct 9, 2020 at 5:44 PM Shinya Okano wrote: > > Hi, > > Regarding the toast_tuple_target parameter of CREATE TABLE, the > documentation says that it only affects External or Extended, but it > actually affects the compression of Main as well. > > The attached patch modifies the documen

Re: Get memory contexts of an arbitrary backend process

2020-10-01 Thread Kasahara Tatsuhito
Hi, On Fri, Sep 25, 2020 at 4:28 PM torikoshia wrote: > Thanks for all your comments, I updated the patch. Thanks for updating the patch. I did a brief test and code review. > I added a shared hash table consisted of minimal members > mainly for managing whether the file is dumped or not. > Some

Re: Get memory contexts of an arbitrary backend process

2020-09-10 Thread Kasahara Tatsuhito
Hi, On Thu, Sep 10, 2020 at 8:53 PM torikoshia wrote: > > On 2020-09-04 21:46, Tomas Vondra wrote: > > On Fri, Sep 04, 2020 at 11:47:30AM +0900, Kasahara Tatsuhito wrote: > >> On Fri, Sep 4, 2020 at 2:40 AM Tom Lane wrote: > >>> Kasahara Tatsuhito writes: &

Re: autovac issue with large number of tables

2020-09-10 Thread Kasahara Tatsuhito
Therefore, we expect this patch [1] to be committed for its original purpose, as well as to improve autovacuum from v14 onwards.Hi, On Fri, Sep 4, 2020 at 7:50 PM Kasahara Tatsuhito wrote: > > Hi, > > On Wed, Sep 2, 2020 at 2:10 AM Kasahara Tatsuhito > wrote: > > > I

Re: autovac issue with large number of tables

2020-09-04 Thread Kasahara Tatsuhito
Hi, On Wed, Sep 2, 2020 at 2:10 AM Kasahara Tatsuhito wrote: > > I wonder if we could have table_recheck_autovac do two probes of the stats > > data. First probe the existing stats data, and if it shows the table to > > be already vacuumed, return immediately. If not, *t

Re: Get memory contexts of an arbitrary backend process

2020-09-03 Thread Kasahara Tatsuhito
On Fri, Sep 4, 2020 at 2:40 AM Tom Lane wrote: > Kasahara Tatsuhito writes: > > Yes, but it's not only for future expansion, but also for the > > usability and the stability of this feature. > > For example, if you want to read one dumped file multiple times and ana

Re: Get memory contexts of an arbitrary backend process

2020-09-03 Thread Kasahara Tatsuhito
Hi, On Thu, Sep 3, 2020 at 3:38 PM torikoshia wrote: > >> - Currently, "the signal transmission for dumping memory > >> information" > >> and "the read & output of dump information " > >> are on the same interface, but I think it would be better to > >> separate them. > >> How about providing the

Re: autovac issue with large number of tables

2020-09-01 Thread Kasahara Tatsuhito
Hi, On Wed, Aug 12, 2020 at 2:46 AM Tom Lane wrote: > So I think Kasahara-san's point is that the shared memory stats collector > might wipe out those costs, depending on how it's implemented. (I've not > looked at that patch in a long time either, so I don't know how much it'd > cut the reader-

Re: Get memory contexts of an arbitrary backend process

2020-08-31 Thread Kasahara Tatsuhito
Hi, On Mon, Aug 31, 2020 at 8:22 PM torikoshia wrote: > As discussed in the thread[1], it'll be useful to make it > possible to get the memory contexts of an arbitrary backend > process. +1 > Attached PoC patch makes pg_get_backend_memory_contexts() > display meory contexts of the specified PID

Re: Creating a function for exposing memory usage of backend process

2020-08-19 Thread Kasahara Tatsuhito
On 2020/08/20 0:01, Tom Lane wrote: > The only situation I could imagine where this would have any use is > where there is long-term (cross-query) bloat in, say, CacheMemoryContext Yeah, in cases where a very large number of sessions are connected to the DB for very long periods of time, the memory

Re: Creating a function for exposing memory usage of backend process

2020-08-07 Thread Kasahara Tatsuhito
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed I tested the latest patch(0007-Adding-a-function-exposing-memory

Re: autovac issue with large number of tables

2020-07-30 Thread Kasahara Tatsuhito
Hi, On Tue, Jul 28, 2020 at 3:49 AM Jim Nasby wrote: > I'm in favor of trying to improve scheduling (especially allowing users > to control how things are scheduled), but that's a far more invasive > patch. I'd like to get something like this patch in without waiting on a > significantly larger e

Re: Creating a function for exposing memory usage of backend process

2020-07-29 Thread Kasahara Tatsuhito
Hi, On Fri, Jul 10, 2020 at 5:32 PM torikoshia wrote: > - whether information for identifying parent-child relation is necessary > or it's an overkill I think it's important to understand the parent-child relationship of the context. Personally, I often want to know the following two things .. -

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-07-10 Thread Kasahara Tatsuhito
Hi, On Wed, Jul 8, 2020 at 9:40 PM Bharath Rupireddy wrote: > One way, we could solve the above problem is that, upon firing the new > foreign query from local backend using the cached connection, > (assuming the remote backend that was cached in the local backed got > killed for some reasons), i

Re: Creating a function for exposing memory usage of backend process

2020-06-26 Thread Kasahara Tatsuhito
Hi, On Fri, Jun 26, 2020 at 3:42 PM Bharath Rupireddy wrote: > While going through the mail chain on relation, plan and catalogue > caching [1], I'm thinking on the lines that is there a way to know the > current relation, plan and catalogue cache sizes? If there is a way > already, please ignor

Re: Creating a function for exposing memory usage of backend process

2020-06-25 Thread Kasahara Tatsuhito
Hi ! On Thu, Jun 18, 2020 at 12:56 PM Fujii Masao wrote: > Agreed. The feature to view how local memory contexts are used in > each process is very useful! +1 > >=# SELECT * FROM pg_stat_get_backend_memory_context(PID); > > I'm afraid that this interface is not convenient when we want to mon

Re: small improvement of the elapsed time for truncating heap in vacuum

2020-02-16 Thread Kasahara Tatsuhito
Hi, On Mon, Feb 17, 2020 at 1:07 PM Masahiko Sawada wrote: > For the patch, we can put also the declaration of ru0 into the loop. Thanks for your reply. Hmm, certainly that it may be better. Fix the v2 patch and attached. Best regards, -- Tatsuhito Kasahara kasahara.tatsuhito _at_ gmail.com

Re: small improvement of the elapsed time for truncating heap in vacuum

2020-02-16 Thread Kasahara Tatsuhito
Hi, On Fri, Feb 14, 2020 at 4:50 PM Fujii Masao wrote: > Regarding the patch, isn't it better to put pg_rusage_init() at the > top of do loop block? If we do this, as a side-effect, we can get > rid of pg_rusage_init() at the top of lazy_truncate_heap(). Thanks for your reply. Yeah, it makes sens

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Kasahara Tatsuhito
On Fri, Feb 7, 2020 at 10:09 PM Fujii Masao wrote: > Pushed! Thanks! Thanks Fujii. -- Tatsuhito Kasahara kasahara.tatsuhito _at_ gmail.com

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Kasahara Tatsuhito
Hi, On Fri, Feb 7, 2020 at 5:02 PM Fujii Masao wrote: > BTW, commit 147e3722f7 causing the issue changed currtid_byreloid() > and currtid_byrelname() so that they also call table_beginscan(). > I'm not sure what those functions are, but probably we should fix them > so that table_beginscan_tid()

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kasahara Tatsuhito
Hi, On Fri, Feb 7, 2020 at 1:27 PM Kyotaro Horiguchi wrote: > It seems that nkeys and key are useless. Since every table_beginscan_* > functions have distinct parameter sets, don't we remove them from > table_beginscan_tid? Yeah, actually, when calling table_beginscan_tid(), nkeys is set to 0 and

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kasahara Tatsuhito
Hi, On Thu, Feb 6, 2020 at 11:01 PM Masahiko Sawada wrote: > > On Thu, 6 Feb 2020 at 19:12, Kasahara Tatsuhito > wrote: > I've tested predicate locking including promotion cases with v3 patch > and it works fine. > > +table_beginscan_tid(Relation rel, Snapshot snaps

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kasahara Tatsuhito
HI, On Thu, Feb 6, 2020 at 3:24 PM Fujii Masao wrote: > > I added a new (but minimal) isolation test for the case of tid scan. > > (v12 and HEAD will be failed this test. v11 and HEAD with my patch > > will be passed) > > Isn't this test scenario a bit overkill? We can simply test that > as fol

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Kasahara Tatsuhito
Hi, On Thu, Feb 6, 2020 at 11:48 AM Andres Freund wrote: > I think it'd be good if we could guard against b) via an isolation > test. It's more painful to do that for a), due to the unreliability of > stats at the moment (we have some tests, but they take a long time). Thanks for your advise, and

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-04 Thread Kasahara Tatsuhito
On Mon, Feb 3, 2020 at 6:20 PM Kasahara Tatsuhito wrote: > Therefore, from v12, Tid scan not only increases the value of > seq_scan, but also acquires a predicate lock. Based on further investigation and Fujii's advice, I've summarized this issue as follows. >From commit 1

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-03 Thread Kasahara Tatsuhito
On Mon, Feb 3, 2020 at 5:33 PM Fujii Masao wrote: > Thanks for explaining that! But heap_beginscan_internal() was really > called in TidScan case? Oh, you are right. Tid Scan started calling table_beginscan from v12 (commit 147e3722f7). So previously(-v11), Tid Scan might never calls heap_beginsca

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-02 Thread Kasahara Tatsuhito
On Mon, Feb 3, 2020 at 4:22 PM Fujii Masao wrote: > On 2020/02/01 16:05, Kasahara Tatsuhito wrote: > > if (scan->rs_base.rs_flags & (SO_TYPE_SEQSCAN | SO_TYPE_SAMPLESCAN)) > > { > > /* > > * Ensure a missing snapsho

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-31 Thread Kasahara Tatsuhito
On Thu, Jan 30, 2020 at 1:49 PM Kyotaro Horiguchi wrote: > At Thu, 30 Jan 2020 13:30:56 +0900, Kasahara Tatsuhito > wrote in > > > TID scan : yes , seq_scan, , > > Here is wrong, because TID scan came to have SO_TYPE_SEQSCAN flags > > from c

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Kasahara Tatsuhito
Hi, On Thu, Jan 30, 2020 at 10:55 AM Kyotaro Horiguchi wrote: > At Wed, 29 Jan 2020 23:24:09 +0900, Fujii Masao > wrote in > > On 2020/01/29 20:06, Kasahara Tatsuhito wrote: > > > Although I'm not sure this behavior is really problem or not, > > > it seems t

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Kasahara Tatsuhito
r not, it seems to me that previous behavior is more prefer. Is it worth to apply to HEAD and v12 branch ? Best regards, 2020年1月27日(月) 14:35 Kasahara Tatsuhito : > > Hi, I noticed that since PostgreSQL 12, Tid scan increments value of > pg_stat_all_tables.seq_scan. (but not seq_tup

Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-26 Thread Kasahara Tatsuhito
Hi, I noticed that since PostgreSQL 12, Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read) The following is an example. CREATE TABLE t (c int); INSERT INTO t SELECT 1; SET enable_seqscan to off; (v12 -) =# EXPLAIN ANALYZE SELECT * FROM t WHERE ctid = '(0,1)';

small improvement of the elapsed time for truncating heap in vacuum

2019-08-12 Thread Kasahara Tatsuhito
Hi, I got following log messages when measured the heap truncating duration in a vacuum. = INFO: "dst": suspending truncate due to conflicting lock request INFO: "dst": truncated 550073 to 101472 pages DETAIL: CPU: user: 0.35 s, system: 4.92