Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-21 Thread Kyotaro Horiguchi
ails’: > describe.c:3420:5: warning: this ‘if’ clause does not guard... > [-Wmisleading-indentation] > if (tableinfo.relam) > ^~ > describe.c:3423:2: note: ...this statement, but the latter is misleadingly > indented as if it were guarded by the ‘if’ > if (res) ^~ regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-21 Thread Kyotaro Horiguchi
sing tools more and less can be invoked as: >more -x4 >less -x4 > to make them show tabs appropriately. But AFAICS the "more" command on CentOS doesn't have -x options nor any option to change tab width and I don't find a document that suggests its existence on o

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-22 Thread Kyotaro Horiguchi
ControlFile->state = DB_SHUTDOWNED_IN_RECOVERY; > > Same as above. IMO it's safer and better to always update the state > (whether the state is DB_IN_ARCHIVE_RECOVERY or not) if > CHECKPOINT_IS_SHUTDOWN flag is passed. That means we may exit recovery mode after ShutdownXLOG

Re: remove more archiving overhead

2022-02-23 Thread Kyotaro Horiguchi
uced links to the same file for the current > > WAL file and the next one. Maybe WAL recyling should use durable_rename() > > instead of durable_rename_excl(). > > Here is an updated patch. Is it intentional that v2 loses the xlogarchive.c part? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-23 Thread Kyotaro Horiguchi
At Tue, 22 Feb 2022 09:59:09 +0100, Daniel Gustafsson wrote in > > On 22 Feb 2022, at 07:14, Kyotaro Horiguchi wrote: > > > Anyway, don't we use the -ftabstop option flag to silence compiler? > > The warning is contradicting our convention. The attached adds that

Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-23 Thread Kyotaro Horiguchi
At Thu, 24 Feb 2022 14:44:56 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 22 Feb 2022 09:59:09 +0100, Daniel Gustafsson wrote > in > > The section in question was added to our docs 22 years ago, to make it > > reflect > > the current operating systems in use m

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-02-23 Thread Kyotaro Horiguchi
ctually WAL did not ended in an incomplete record. I think FinishWalRecover is the last place to do that. (But it could be earlier.) regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-02-24 Thread Kyotaro Horiguchi
At Thu, 24 Feb 2022 16:26:42 +0900 (JST), Kyotaro Horiguchi wrote in > So, actually WAL did not ended in an incomplete record. I think > FinishWalRecover is the last place to do that. (But it could be > earlier.) After some investigation, I finally concluded that we should reset abor

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Kyotaro Horiguchi
ould make > sense and amend my patch. I don't think there is. I remember sometimes wanted ids on some sections(x.x) and items(x.x.x or lower) (or even clauses, ignoring costs:p) FWIW in that perspecive, there's no requirement from me that it should be human-readable. I'm fine wit

Re: Buffer Manager and Contention

2022-02-24 Thread Kyotaro Horiguchi
rst part, I think Yura's patch works. On the second point, Yura already showed it gives a certain amount of gain if we do that. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Buffer Manager and Contention

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 10:20:25 +0900 (JST), Kyotaro Horiguchi wrote in > (I added Yura, as the author of a related patch) > > At Thu, 24 Feb 2022 12:58:23 +, Simon Riggs > wrote in > > Thinking about poor performance in the case where the data fits in > > RAM, bu

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Tue, 22 Feb 2022 17:44:01 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 22 Feb 2022 01:59:45 +0900, Fujii Masao > wrote in > > > > > Of the following, I think we should do (a) and (b) to make future > > > backpatchings easier. > > > a) Use Red

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 15:31:12 +0900 (JST), Kyotaro Horiguchi wrote in > While making patches for v12, I see a test failure of pg_rewind for > uncertain reason. I'm investigating that but I post this for > discussion. Hmm. Too stupid. Somehow I overly removed the latchet

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 16:06:31 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 25 Feb 2022 15:31:12 +0900 (JST), Kyotaro Horiguchi > wrote in > > While making patches for v12, I see a test failure of pg_rewind for > > uncertain reason. I'm investigating t

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 16:47:01 +0900 (JST), Kyotaro Horiguchi wrote in > So, this is the patches for pg12-10. 11 can share the same patch with > 12. 10 has differences in two points. > > 10 has ControlFile->prevCheckPoint. > > The DETAILS of the "recovery restar

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-25 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 16:52:30 +0900 (JST), Kyotaro Horiguchi wrote in > Ugh! Wait for a moment. Something's wrong. Sorry, what is wrong was my working directory. It was broken by my bogus operation. All the files apply corresponding versions correctly. regards. -- Kyotaro Horiguchi

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 00:04:55 -0800, Andres Freund wrote in > Why don't you just use LockBufHdr/UnlockBufHdr? FWIW, v2 looked fine to me in regards to this point. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-02-27 Thread Kyotaro Horiguchi
in an area that's already > a lot sensitive. That sounds like we should reject the patch as we don't agree to its objective. If someday end-of-recovery checkpoints functionally diverge from shutdown checkpoints but leave (somehow) the transition alone, we may visit this again but it wou

Re: definition of CalculateMaxmumSafeLSN

2022-02-28 Thread Kyotaro Horiguchi
ut eventually gone. So it should be removed. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-02-28 Thread Kyotaro Horiguchi
especially in the documentation. I'm not sure how we should treat "WAL log", though. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: In-placre persistance change of a relation

2022-02-28 Thread Kyotaro Horiguchi
init fork may be stale. However, AFAICS init-fork is write-once fork so I don't think that actually matters. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From 420a9d9a0dae3bcfb1396c14997624ad67a3e557 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Wed, 11 Nov 20

Re: Allow async standbys wait for sync replication

2022-02-28 Thread Kyotaro Horiguchi
ut doesn't that makes walsender falsely believes it have caught up to the bleeding edge of WAL? > replicated LSN. TBH there are probably other things that need to be > considered (e.g., how do we ensure that the WAL sender sends the rest once > it is replicated?), but I still thi

Re: In-placre persistance change of a relation

2022-03-01 Thread Kyotaro Horiguchi
At Tue, 01 Mar 2022 14:14:13 +0900 (JST), Kyotaro Horiguchi wrote in > - Removed the default case in smgr_desc since it seems to me we don't > assume out-of-definition values in xlog records elsewhere. Stupid. The complier on the CI environemnt complains for uninitialized va

Re: Make mesage at end-of-recovery less scary.

2022-03-01 Thread Kyotaro Horiguchi
in rather large duplicate code. I could have ValidXLogRecordHeader handle the partial header case but it seems to me complex. So in this version I split the xl_tot_len part of ValidXLogRecordHeader into ValidXLogRecordLength. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >

Re: more descriptive message for process termination due to max_slot_wal_keep_size

2022-03-01 Thread Kyotaro Horiguchi
At Tue, 04 Jan 2022 10:29:31 +0900 (JST), Kyotaro Horiguchi wrote in > So what do you say if I propose the following? > > LOG: terminating process %d to release replication slot \"%s\" > because its restart_lsn %X/%X exceeds the limit %X/%X > HINT: Y

Re: Add the replication origin name and commit-LSN to logical replication worker errcontext

2022-03-01 Thread Kyotaro Horiguchi
by modulating it, but it still looks odd. CONTEXT: 時刻(非設定)、トランザクション 2352314 内で "TRUNCATE" でのリモートデータの処理中 Rather, I'd prefer simply to list the attributes. CONTEXT: processing remote data during "MESSAGE". Transaction (unknown). Time (unknown), replication target relation (unknown), column (unknown) CONTEXT: "MESSAGE"でのリモートデータの処理中. トランザクション (不明). 時刻 (不明), レプリケーション対象リレーション (不明), column (不明) regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-01 Thread Kyotaro Horiguchi
At Thu, 20 Jan 2022 17:19:04 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 20 Jan 2022 15:07:22 +0900 (JST), Kyotaro Horiguchi > wrote in > CI now likes this version for all platforms. An xlog.c refactoring happend recently hit this. Just rebased on the change. regards. --

Re: pg_stop_backup() v2 incorrectly marked as proretset

2022-03-02 Thread Kyotaro Horiguchi
PG_LSNOID, -1, 0); I think we can use get_call_resuilt_type here. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Two noncritical bugs of pg_waldump

2022-03-02 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 10:48:47 -0800, Andres Freund wrote in > Hi, > > On 2022-02-14 18:18:47 +0900, Kyotaro Horiguchi wrote: > > > If I give an empty file to the tool it complains as the follows. > > > > > > > pg_waldump: fatal: could not read

Re: more descriptive message for process termination due to max_slot_wal_keep_size

2022-03-02 Thread Kyotaro Horiguchi
At Wed, 02 Mar 2022 15:37:19 +0900 (JST), Kyotaro Horiguchi wrote in > The CI was confused by the mixed patches for multiple PG versions. In > this version the patchset for master are attached as .patch and that > for PG13 as .txt. Yeah It is of course the relevant check should

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-02 Thread Kyotaro Horiguchi
At Wed, 02 Mar 2022 16:59:09 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 20 Jan 2022 17:19:04 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Thu, 20 Jan 2022 15:07:22 +0900 (JST), Kyotaro Horiguchi > > wrote in > > CI now likes this version for all p

Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Kyotaro Horiguchi
mem)->ckpt_flags) & CHECKPOINT_REQUESTED)) I don't particularly object to this, but I don't think that change makes the code significantly easier to read either. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Changing "Hot Standby" to "hot standby"

2022-03-02 Thread Kyotaro Horiguchi
n a sentsnce. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-02 Thread Kyotaro Horiguchi
ta is present, rather than waiting for new WAL data > to become available, default behaviour could be anything you like. > > [2] > https://www.postgresql.org/message-id/CALj2ACUtqWX95uAj2VNJED0PnixEeQ%3D0MEzpouLi%2Bzd_iTugRA%40mail.gmail.com > I've added a new function read_local_xlog_page_2 (similar to > read_local_xlog_page but works in wait and no wait mode) and the > callers can specify whether to wait or not wait using private_data. > Actually, I wanted to use the private_data structure of > read_local_xlog_page but the logical decoding already has context as > private_data, that is why I had to have a new function. I know it > creates a bit of duplicate code, but its cleaner than using > backend-local variables or additional flags in XLogReaderState or > adding wait/no-wait boolean to page_read callback. Any other > suggestions are welcome here. > > With this, I'm able to have wait/no wait versions for any functions. > But for now, I'm having wait/no wait for two functions > (pg_get_wal_records_info and pg_get_wal_stats) for which it makes more > sense. > > Regards, > Bharath Rupireddy. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Why do spgbuildempty(), btbuildempty(), and blbuildempty() use smgrwrite()?

2022-03-02 Thread Kyotaro Horiguchi
em onto shared buffers. They are cold images of init-fork, which is actually no-use for active servers. Rather I'd like to move brinbuildempty out of shared buffers considering one of my proposing patches.. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Kyotaro Horiguchi
At Thu, 3 Mar 2022 10:27:10 +0900, Michael Paquier wrote in > On Thu, Mar 03, 2022 at 09:39:37AM +0900, Kyotaro Horiguchi wrote: > > At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy > > wrote in > >> I don't think that's useful. Being in LogCheckpo

Re: Changing "Hot Standby" to "hot standby"

2022-03-02 Thread Kyotaro Horiguchi
onnect to Yes. Isn't it the right form of a sentence? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: shared-memory based stats collector

2022-03-03 Thread Kyotaro Horiguchi
atLocal_ArchiverCounts PgStatLocal_BgWriterCounts PgStatShared_TableStats PgStatShared_FunctionStats PgStatShared_GlobalCounts PgStatShared_ArchiverCounts PgStatShared_BgWriterCounts PgStatLocal_GlobalCounts somewhat looks odd, but doesn't matter much, maybe. > I'd like that get resolved first because I think that'd allow commiting the > prepatory split and reordering patches. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-03 Thread Kyotaro Horiguchi
At Wed, 02 Mar 2022 19:31:24 +0900 (JST), Kyotaro Horiguchi wrote in > A function added to Util.pm used perl2host, which has been removed > recently. And same function contained a maybe-should-have-been-removed line which makes Windows build unhappy. This should make all platforms in

Re: Make mesage at end-of-recovery less scary.

2022-03-03 Thread Kyotaro Horiguchi
eck is necessary here to ensure that we enter the "Need to > reassemble >* record" code path below; otherwise we might fail to apply >* ValidXLogRecordHeader at all. >*/ > record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ); &g

Re: pg_stop_backup() v2 incorrectly marked as proretset

2022-03-03 Thread Kyotaro Horiguchi
round without any undefs, so it looks like people like > that way of doing things. I'm one of them. Not unliking #undef, though. It seems to me the name "PG_STOP_BACKUP_V2_COLS" alone is specific enough for the purpose of avoiding misuse. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

false failure of test_docoding regression test

2022-03-03 Thread Kyotaro Horiguchi
to wait for process termination at the time. postgres_fdw does that in regression test. Thoughts? Simliar use is found in temp-schema-cleanup. There's another possible instability between s2_advisory and s2_check_schema but this change alone reduces the chance for false failures. The

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-03-03 Thread Kyotaro Horiguchi
At Mon, 14 Feb 2022 14:52:15 +0900 (JST), Kyotaro Horiguchi wrote in > In this version , 0001 gets one fix and two comment updates. While the disucssion on back-patching of 0001 proceeding on another branch, the main patch iself gets looks like as if rotten on CF-App. So I rebased v10 on

Re: Doc about how to set max_wal_senders when setting minimal wal_level

2022-03-03 Thread Kyotaro Horiguchi
his documentaiotn. Addtion to that, if we write this for max_wal_senders, archive_mode will deserve the similar tips but I think it is too verbose. In short, I think I would not add that description at all. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-03 Thread Kyotaro Horiguchi
Thanks to look this! At Fri, 4 Mar 2022 13:51:12 +0900, Michael Paquier wrote i n > On Fri, Mar 04, 2022 at 09:10:48AM +0900, Kyotaro Horiguchi wrote: > > And same function contained a maybe-should-have-been-removed line > > which makes Windows build unhappy. > > >

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-03 Thread Kyotaro Horiguchi
ace tablespaces. While it is apparently incovenient for general use, there might be a case where we want to know whether the tablespace is in-place or not. So I'm not sure which is better.. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-03 Thread Kyotaro Horiguchi
At Fri, 04 Mar 2022 16:41:03 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 4 Mar 2022 15:44:22 +0900, Michael Paquier > wrote in > > The use may be limited to any automated testing and > > allow_in_place_tablespaces is a developer GUC, still it seems to me > > t

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-04 Thread Kyotaro Horiguchi
At Fri, 04 Mar 2022 16:54:49 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 04 Mar 2022 16:41:03 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Fri, 4 Mar 2022 15:44:22 +0900, Michael Paquier > > wrote in > > > The use may be limited t

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-04 Thread Kyotaro Horiguchi
At Fri, 04 Mar 2022 17:28:45 +0900 (JST), Kyotaro Horiguchi wrote in > > By the way, regardless of the patch, I got an error from pg_basebackup > > for an in-place tablespace. pg_do_start_backup calls readlink > > believing pg_tblspc/* is always a symlink. > > > &g

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-06 Thread Kyotaro Horiguchi
At Fri, 4 Mar 2022 23:26:43 +1300, Thomas Munro wrote in > On Fri, Mar 4, 2022 at 10:04 PM Kyotaro Horiguchi > wrote: > > And I made a quick hack on do_pg_start_backup. And I found that > > pg_basebackup copies in-place tablespaces under the *current > > directory*

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-07 Thread Kyotaro Horiguchi
So the new framework has been dropped in this version. The second test is removed as it is irrelevant to this bug. In this version the patch is a single file that contains the test. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From 43bb3ba8900edd53a1feb0acb1a72bdc22bb1627

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-07 Thread Kyotaro Horiguchi
0 nbase.tar 2022/03/08 09:53 pg_wal 1 File(s) 0 bytes 3 Dir(s) 171,920,613,376 bytes free regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-07 Thread Kyotaro Horiguchi
At Tue, 8 Mar 2022 10:28:46 +0900, Michael Paquier wrote in > On Tue, Mar 08, 2022 at 10:06:50AM +0900, Kyotaro Horiguchi wrote: > > At Tue, 8 Mar 2022 10:39:06 +1300, Thomas Munro > > wrote in > >> Thanks, you're right. Test on a Win10 VM. Here's a

Re: Naming of the different stats systems / "stats collector"

2022-03-08 Thread Kyotaro Horiguchi
In short, deal with this > mainly in documentation/comments and implementation details but leave the > public facing naming alone. > > David J. If we could, I like the namings like pg_metrics.process, pg_metrics.replication, pg_progress.vacuum, pg_progress.basebackup, and pg_stats.database, pg_stats.user_tables.. With such eyes, it looks somewhat odd that pg_stat_* views are belonging to the pg_catalog namespace. If we had system table-aliases, people who insist on the good-old names can live with that. Even if there isn't, we can instead provide views with the old names. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-10 Thread Kyotaro Horiguchi
cords_info('0/100', '0/1000000'); finds the first record since the start lsn and show it. - pg_get_wal_records_info('0/100', '0/130'); finds the first record since the start lsn then show records up to the end-lsn. And about pg_get_raw_wal_record(). I don't see any use-case of the function alone on SQL interface. Even if we need to inspect broken WAL files, it needs profound knowledge of WAL format and tools that doesn't work on SQL interface. However like pageinspect, if we separate the WAL-record fetching and parsing it could be thought as useful. pg_get_wal_records_info woule be like: SELECT * FROM pg_walinspect_parse(raw) FROM (SELECT * FROM pg_walinspect_get_raw(start_lsn, end_lsn)); And pg_get_wal_stats woule be like: SELECT * FROM pg_walinpect_stat(pg_walinspect_parse(raw)) FROM (SELECT * FROM pg_walinspect_get_raw(start_lsn, end_lsn))); Regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-10 Thread Kyotaro Horiguchi
Sorry, some minor non-syntactical corrections. At Fri, 11 Mar 2022 11:38:22 +0900 (JST), Kyotaro Horiguchi wrote in > I played with this a bit, and would like to share some thoughts on it. > > It seems to me too rigorous that pg_get_wal_records_info/stats() > reject future LSNs as

Re: BufferAlloc: don't take two simultaneous locks

2022-03-10 Thread Kyotaro Horiguchi
ction) other than HASH_ASSIGN needs this assertion and no need for checking both, maybe only for element would be enough. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: BufferAlloc: don't take two simultaneous locks

2022-03-10 Thread Kyotaro Horiguchi
At Fri, 11 Mar 2022 15:30:30 +0900 (JST), Kyotaro Horiguchi wrote in > Thanks! I looked into dynahash part. > > struct HASHHDR > { > - /* > - * The freelist can become a point of contention in high-concurrency > hash > > Why did you move around the f

Re: BufferAlloc: don't take two simultaneous locks

2022-03-11 Thread Kyotaro Horiguchi
At Fri, 11 Mar 2022 15:49:49 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 11 Mar 2022 15:30:30 +0900 (JST), Kyotaro Horiguchi > wrote in > > Thanks! I looked into dynahash part. Then I looked into bufmgr part. It looks fine to me but I have some comments on c

Re: BufferAlloc: don't take two simultaneous locks

2022-03-13 Thread Kyotaro Horiguchi
At Fri, 11 Mar 2022 11:30:27 +0300, Yura Sokolov wrote in > В Пт, 11/03/2022 в 15:30 +0900, Kyotaro Horiguchi пишет: > > At Thu, 03 Mar 2022 01:35:57 +0300, Yura Sokolov > > wrote in > > > В Вт, 01/03/2022 в 10:24 +0300, Yura Sokolov пишет: > > > > Ok, her

Re: BufferAlloc: don't take two simultaneous locks

2022-03-13 Thread Kyotaro Horiguchi
At Fri, 11 Mar 2022 12:34:32 +0300, Yura Sokolov wrote in > В Пт, 11/03/2022 в 15:49 +0900, Kyotaro Horiguchi пишет: > > At Fri, 11 Mar 2022 15:30:30 +0900 (JST), Kyotaro Horiguchi > > > BufTableDelete(BufferTag *tagPtr, uint32 hashcode, bool > > reuse) > > >

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-13 Thread Kyotaro Horiguchi
At Fri, 11 Mar 2022 15:39:13 -0500, Robert Haas wrote in > On Thu, Mar 10, 2022 at 9:38 PM Kyotaro Horiguchi > wrote: > > It seems to me too rigorous that pg_get_wal_records_info/stats() > > reject future LSNs as end-LSN and I think WARNING or INFO and stop at > > the

Re: Allow async standbys wait for sync replication

2022-03-13 Thread Kyotaro Horiguchi
ng existing unreplayed records after reconnection. But, putting aside how to remember that LSN, if that happens at a segment boundary, the async replica may run into the similar situation with the missing-contrecord case. But standby cannot insert any original record to get out from that situation. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Allow async standbys wait for sync replication

2022-03-13 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 11:30:02 +0900 (JST), Kyotaro Horiguchi wrote in > At Sat, 12 Mar 2022 14:33:32 -0800, Nathan Bossart > wrote in > > On Tue, Mar 08, 2022 at 06:01:23PM -0800, Andres Freund wrote: > > > To me it's architecturally the completely wrong direction

Re: BufferAlloc: don't take two simultaneous locks

2022-03-13 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 09:39:48 +0900 (JST), Kyotaro Horiguchi wrote in > I'll examine the possibility to resolve this... The existence of nfree and nalloc made me confused and I found the reason. In the case where a parittion collects many REUSE-ASSIGN-REMOVEed elemetns from other pa

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-13 Thread Kyotaro Horiguchi
At Thu, 17 Feb 2022 17:29:15 +, Jacob Champion wrote in > On Tue, 2022-02-15 at 15:16 +0900, Kyotaro Horiguchi wrote: > > (This needs rebasing) > > Done in v6, attached. Thanks! > > # I forgot to mention that, the test fails for me even without the > > # change

Re: BufferAlloc: don't take two simultaneous locks

2022-03-14 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 09:15:11 +0300, Yura Sokolov wrote in > В Пн, 14/03/2022 в 14:31 +0900, Kyotaro Horiguchi пишет: > > I'd like to ask you to remove nalloced from partitions then add a > > global atomic for the same use? > > I really believe it should be global.

Re: BufferAlloc: don't take two simultaneous locks

2022-03-14 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 17:12:48 +0900 (JST), Kyotaro Horiguchi wrote in > Then, I tried the same with the patch, and I am surprized to see that > the rise of the number of newly allocated elements didn't stop and > went up to 511 elements after the 100 seconds run. So I found tha

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Kyotaro Horiguchi
; pg_strtouint64() is a wrapper around strtoull/strtoul/_strtoui64, but > it seems no longer necessary to have this indirection. .. >type definition int64/uint64. For that, add new macros strtoi64() and >strtou64() in c.h as thin wrappers around strtol()/strtoul() or >strtoll()/stroull(). This makes these functions available everywhere regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-14 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 17:37:40 -0400, Robert Haas wrote in > On Mon, Mar 7, 2022 at 3:39 AM Kyotaro Horiguchi > wrote: > > So the new framework has been dropped in this version. > > The second test is removed as it is irrelevant to this bug. > > > > In this vers

Re: BufferAlloc: don't take two simultaneous locks

2022-03-15 Thread Kyotaro Horiguchi
Thanks for the new version. At Tue, 15 Mar 2022 08:07:39 +0300, Yura Sokolov wrote in > В Пн, 14/03/2022 в 14:57 +0300, Yura Sokolov пишет: > > В Пн, 14/03/2022 в 17:12 +0900, Kyotaro Horiguchi пишет: > > > At Mon, 14 Mar 2022 09:15:11 +0300, Yura Sokolov > > > wr

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-03-15 Thread Kyotaro Horiguchi
At Tue, 15 Mar 2022 12:19:47 +0530, Bharath Rupireddy wrote in > On Fri, Mar 4, 2022 at 10:40 AM Kyotaro Horiguchi > wrote: > 0001 - I don't think you need to do this as UpdateControlFile > (update_controlfile) will anyway update it, no? > + /* Update control fil

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-03-15 Thread Kyotaro Horiguchi
At Tue, 15 Mar 2022 18:26:26 +0900, Michael Paquier wrote in > On Tue, Mar 15, 2022 at 05:23:40PM +0900, Kyotaro Horiguchi wrote: > > At Tue, 15 Mar 2022 12:19:47 +0530, Bharath Rupireddy > > wrote in > >> On Fri, Mar 4, 2022 at 10:40 AM Kyotaro Horiguchi > >

Possible corruption by CreateRestartPoint at promotion

2022-03-15 Thread Kyotaro Horiguchi
.org/message-id/20220316.091913.806120467943749797.horikyota.ntt%40gmail.com [2] https://www.postgresql.org/message-id/7bfad665-db9c-0c2a-2604-9f54763c5f9e%40oss.nttdata.com [3] https://www.postgresql.org/message-id/20220222.174401.765586897814316743.horikyota.ntt%40gmail.com -- Kyotaro Horiguchi

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-03-15 Thread Kyotaro Horiguchi
stop reading at WAL LSN RECPTR\n")); > > > > Mmm.. "WAL LSN RECPTR" looks strange to me. In the first place I > don't think "RECPTR" is a user-facing term. Doesn't something like the > follows work? > > > > + print

Unhyphenation of crash-recovery

2022-03-15 Thread Kyotaro Horiguchi
es. regards. [1] https://www.postgresql.org/message-id/20220316.091913.806120467943749797.horikyota.ntt%40gmail.com -- Kyotaro Horiguchi NTT Open Source Software Center >From 751acd2510e036a5c484e5768dd9e4e85cd8b2cf Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Fri, 4 Mar 2022 13:53:38 +0900

Reword "WAL location" as "WAL LSN"

2022-03-15 Thread Kyotaro Horiguchi
n't something like the > follows work? > > > > + printf(_(" -e, --end=WAL-LSN stop reading at WAL-LSN\n")); > > > > In some changes in this patch shorten the main message text of > fprintf-ish functions. That makes the succeeding parame

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-03-15 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 09:19:13 +0900 (JST), Kyotaro Horiguchi wrote in > In short, I split out the two topics other than checkpoint log to > other threads. So, this is about the main topic of this thread, adding LSNs to checkpint log. Other topics have moved to other treads [1], [2] , [

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-15 Thread Kyotaro Horiguchi
, as we know that all those tablespaces stay in pg_tblspc/ so one > > can make the difference with normal tablespaces more easily. The > > barrier is thin, though :p > > Sounds good to me. +1. Desn't the doc need to mention that? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: BufferAlloc: don't take two simultaneous locks

2022-03-15 Thread Kyotaro Horiguchi
At Tue, 15 Mar 2022 13:47:17 +0300, Yura Sokolov wrote in > В Вт, 15/03/2022 в 16:25 +0900, Kyotaro Horiguchi пишет: > > Hmm. v8 returns stashed element with original patition index when the > > element is *not* reused. But what I saw in the previous test runs is > > t

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-15 Thread Kyotaro Horiguchi
_min with FrozenTransactionId or > InvalidTransactionId respectively during tuple conversion to 64xid. > > Reviews are welcome as always! Thanks! My pleasure. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Corruption during WAL replay

2022-03-15 Thread Kyotaro Horiguchi
At Tue, 15 Mar 2022 12:44:49 -0400, Robert Haas wrote in > On Wed, Jan 26, 2022 at 3:25 AM Kyotaro Horiguchi > wrote: > > The attached is the fixed version and it surely works with the repro. > > Hi, > > I spent the morning working on this patch and came up with t

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-15 Thread Kyotaro Horiguchi
At Tue, 15 Mar 2022 21:41:49 +, Jacob Champion wrote in > On Mon, 2022-03-14 at 15:30 +0900, Kyotaro Horiguchi wrote: > > t/003_sslinfo.pl ... 1/? # Tests were run but no plan was declared and > > done_testing() was not seen. > > # Looks like your test exited with 2

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 15:56:02 +0900 (JST), Kyotaro Horiguchi wrote in > Mmm I'm not sure how come I didn't noticed that, master also fails > for me fo the same reason. In the past that may fail when valid > clinent-certs exists in the users ~/.postgresql but I believe th

Out-of-tree certificate interferes ssltest

2022-03-16 Thread Kyotaro Horiguchi
I have a cert file in my home direcotory. regareds. -- Kyotaro Horiguchi NTT Open Source Software Center >From 308b55f06907ccaf4ac5669daacf04fea8a18fe1 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Wed, 16 Mar 2022 16:20:46 +0900 Subject: [PATCH v1] Prevent out-of-tree certificates from interf

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 15:42:52 +0900, Michael Paquier wrote in > On Wed, Mar 16, 2022 at 10:34:15AM +0900, Kyotaro Horiguchi wrote: > > +1. Desn't the doc need to mention that? > > Yes, I agree that it makes sense to add a note, even if > allow_in_place_tablespaces is a de

Re: Standby got invalid primary checkpoint after crashed right after promoted.

2022-03-16 Thread Kyotaro Horiguchi
increase the chance CreateRestartPoint skip update ControlFile and to > simulate a crash, > the patch 0001 is needed. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Standby got invalid primary checkpoint after crashed right after promoted.

2022-03-16 Thread Kyotaro Horiguchi
181487576617583.horikyota.ntt%40gmail.com<https://www.postgresql.org/message-id/flat/20220316.102444.2193181487576617583.horikyota@gmail.com> Glad to hear that. Thanks for checking it! regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Possible corruption by CreateRestartPoint at promotion

2022-03-16 Thread Kyotaro Horiguchi
Just for the record. An instance of the corruption showed up in this mailing list [1]. [1] https://www.postgresql.org/message-id/flat/9EB4CF63-1107-470E-B5A4-061FB9EF8CC8%40outlook.com regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: BufferAlloc: don't take two simultaneous locks

2022-03-16 Thread Kyotaro Horiguchi
At Wed, 16 Mar 2022 14:11:58 +0300, Yura Sokolov wrote in > В Ср, 16/03/2022 в 12:07 +0900, Kyotaro Horiguchi пишет: > > At Tue, 15 Mar 2022 13:47:17 +0300, Yura Sokolov > > wrote in > > In v7, HASH_ENTER returns the element stored in DynaHashReuse using > > the

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-16 Thread Kyotaro Horiguchi
that record by specifying -s 0/128 -e 0/19a, but not for -e 0/199. # I personally think the current behavior is fine, though.. It still suggests unspecifiable end-LSN.. > select * from pg_get_wal_records_info('4/4B28EB68', '4/4C60'); > ERROR: cannot a

Re: Unhyphenation of crash-recovery

2022-03-17 Thread Kyotaro Horiguchi
At Thu, 17 Mar 2022 07:42:42 +0100, Peter Eisentraut wrote in > On 16.03.22 02:25, Kyotaro Horiguchi wrote: > > Hello, this is a derived topic from [1], summarized as $SUBJECT. > > This just removes useless hyphens from the words > > "(crash|emergency)-recovery".

Re: Out-of-tree certificate interferes ssltest

2022-03-17 Thread Kyotaro Horiguchi
ler and cleaner > patch. > > Thoughts? Ah! I didn't have a thought that we can specify the same parameter twice. It looks like clean and robust. $default_ssl_connstr contains all required options in PQconninfoOptions[]. The same method worked for 003_sslinfo.pl, too. (of course). regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-17 Thread Kyotaro Horiguchi
on it? > > Right, I see your point. The existing description is not wrong > either. Fine by me to just drop all that. +1. Sorry for my otiose comment.. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: XID formatting and SLRU refactorings

2022-03-17 Thread Kyotaro Horiguchi
e change to some inappropriate places? - elog(DEBUG2, "deleted page from block %u has safexid %u", - blkno, opaque->btpo_level); + elog(DEBUG2, "deleted page from block %u has safexid %llu", + blkno, (XID_TYPE) opaque->btpo_level); regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Corruption during WAL replay

2022-03-17 Thread Kyotaro Horiguchi
T. 11-10: A hunk fails due to the context difference due to an additional member tempNamespaceId of PGPROC. regards. -- Kyotaro Horiguchi NTT Open Source Software Center From c88858d3e5681005ba0396b7e7ebcde4322b3308 Mon Sep 17 00:00:00 2001 From: Robert Haas Date: Tue, 15 Mar 2022

Re: XID formatting and SLRU refactorings

2022-03-17 Thread Kyotaro Horiguchi
At Fri, 18 Mar 2022 10:20:08 +0900 (JST), Kyotaro Horiguchi wrote in > "(XID_TYPE) x" is actually equivalent to "(long long) x" here, but the > point here is "%llu in format string accepts (long long)" so we should Of course it's the typo of "

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-18 Thread Kyotaro Horiguchi
cure-common.h itself doesn't need the include. Instead, fe-secure-openssl.c and fe-secure-common.c needs the include. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Replication slot drop message is sent after pgstats shutdown.

2022-03-18 Thread Kyotaro Horiguchi
[1] https://www.postgresql.org/message-id/20220304.113347.2105652521035137491.horikyota@gmail.com regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Make mesage at end-of-recovery less scary.

2022-03-21 Thread Kyotaro Horiguchi
At Mon, 21 Mar 2022 17:01:19 -0700, Andres Freund wrote in > Patch currently fails to apply, needs a rebase: > http://cfbot.cputube.org/patch_37_2490.log Thanks for noticing me of that. Rebased to the current HEAD. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

  1   2   3   4   5   6   7   8   9   10   >